AI-assisted development, safely

Using Claude Code, Copilot, or ChatGPT to build your research software

Odds are you used an AI coding assistant to build significant portions of the code you’re publishing. That’s fine — JOSS’s January 2026 policy statement explicitly welcomes AI-assisted development, provided you disclose it and the software genuinely serves the research community.

This chapter is the practical version: how to work with an AI assistant in a way that produces JOSS-publishable software, and how to disclose the usage in a way that satisfies the review.

The mental model

An AI coding assistant is not a co-author. It is a very fast, extremely uneven collaborator who:

  • Produces syntactically correct code at high volume.
  • Makes plausible but sometimes wrong decisions about libraries, defaults, and architecture.
  • Has no memory across sessions, no stake in your project, and no accountability.
  • Doesn’t know your data. Doesn’t know your users. Doesn’t know your scientific goals.

You keep the intellectual and moral responsibility for the code, no matter how much of it was typed by the assistant. The JOSS review is a check on your judgment, not on the assistant’s output volume.

What “human intellectual contribution” actually looks like

JOSS’s updated criteria emphasise these markers of human contribution. Aim to make each of them visible in your repo:

  1. Problem framing. The Statement of need in paper.md is your framing, not the assistant’s. Write it (or heavily rewrite what the assistant drafted) in your own voice.
  2. Key design decisions. Architectural choices (what your package’s public API looks like, how modules are divided, what abstraction the user interacts with) should be your calls. The assistant can propose options; you decide.
  3. Sustained development over time. A repo with 8 months of commits by a human, some assisted by AI, looks nothing like a repo generated in a weekend by an assistant. Regular commits, code review, iterative refinement — the assistant doesn’t fake these.
  4. Open-source practices (tests, documentation, licensing, versioned releases, transparent issue tracking). The assistant is genuinely helpful with tests and documentation — use it there.
  5. Peer engagement. Merged PRs from other people (advisors, students, external contributors), issues opened by real users, changelog entries with rationale. Signals a real collaborative project, not a solo AI-driven exercise.

If a reviewer flips through your git history and sees eight months of thoughtful commits with meaningful messages, contributions from multiple people, tests added alongside features, and PRs from external users, they will not be worried about how much AI you used.

Concrete practices that work

Use the assistant for volume, keep the judgment for yourself

  • Yes: “Write a test for this function that covers these three edge cases I want to make sure we handle.”
  • Yes: “Refactor this 200-line function into three smaller ones — keep the public signature the same.”
  • Yes: “Write a docstring for this function following NumPy style.”
  • Yes: “Set up a .github/workflows/tests.yml matrix for Python 3.10-3.12 on Ubuntu and macOS.”
  • No: “Decide how my package should be structured.”
  • No: “Choose which functions to expose in the public API.”
  • No: “Design the class hierarchy.”

The pattern: assistant executes, you decide. When you find yourself accepting an assistant’s design suggestion without pushback, stop and ask whether you actually agree, or whether you’re just tired.

Commit messages that show your reasoning

Instead of accepting the assistant’s default commit message (“Add feature X”), write commit messages that explain why the change was made and what trade-offs you weighed. A repo full of "fix bug" and "add tests" commits reads as low-effort AI output; a repo full of "nan_handling: switch to per-kind dispatch to close the bilinear-resample-of-QA bug" reads as a human thinking hard.

The extra minute per commit compounds into a highly credible development history.

Don’t push AI-generated commit attribution to a public repo

Some AI coding assistants add a Co-Authored-By: trailer to commits by default. Turn this off for public research repos:

  • If you’re using Claude Code: don’t accept the default “Co-Authored-By: Claude …” trailer at commit time.
  • If you’re using GitHub Copilot: no automatic trailer, but be mindful of what you paste from chat.
  • If you use conventional commit trailers, keep them limited to actual human co-authors.

Why: the GitHub Contributors widget on your repo’s homepage counts by commit author. A “Claude” contributor showing up next to your students and advisors is a bad look for a research software artifact — and it’s exactly what reviewers point at when they’re worried about human contribution.

If you’ve already committed with those trailers, you can rewrite history with git-filter-repo (see Common pitfalls) — but it’s a big-hammer operation, so don’t leave it lying around.

Document the split visibly

In the Acknowledgements of paper.md, be honest about the split:

Code development since [date] was accelerated by [tool] ([model versions]),
which assisted with code generation, refactoring, and drafting. All
architectural decisions were made by the human authors, who reviewed and
validated every AI-generated artefact before commit.

This meets JOSS’s three specific disclosure items: tools/models used, nature and scope of assistance, and confirmation of human review/validation.

Keep the sentence tight. “Substantially accelerated” is fine. “Substantially written by” is a red flag — even if it’s true.

The JOSS disclosure requirements, exact wording

From joss.readthedocs.io/en/latest/submitting.html, AI-assistance disclosure requires:

  • Tools/models and versions used. (“Claude Code, Opus 4.5-4.7 and Sonnet 4.5-4.6 models” is specific enough; “Claude” is not.)
  • Nature and scope of assistance: e.g., code generation, refactoring, test scaffolding, copy-editing, drafting.
  • Confirmation of review: Authors must assert that human authors reviewed, edited, validated all AI-assisted outputs.

Write this into the Acknowledgements of paper.md (not a separate “AI Statement” section). One sentence per item; three sentences total.

Things to avoid

  • Accepting long stretches of code you don’t read. If you didn’t read it, you can’t debug it, and JOSS reviewers will hit bugs you didn’t know were there.
  • Letting the assistant write your Statement of need. It’s your framing. Every generic phrase (“state-of-the-art,” “innovative,” “comprehensive”) that leaks into your paper.md makes the paper feel machine-written.
  • Pasting your API keys or private data into a chat window. Once it’s uploaded, you have to assume it’s persisted somewhere. Rotate the key regardless.
  • Committing AI-generated changes without running tests. The assistant will happily “fix” your code by silently breaking a subtle invariant. Tests catch this. Nothing else does.
  • Skipping code review by other humans. Even one advisor / labmate / collaborator reviewing a PR provides more oversight than 100 hours of AI assistance.

The fast path — drop a prebuilt AGENTS.md in your repo

Rather than teaching your AI assistant the rules above one conversation at a time, you can hand it a compact instruction file that captures all of this — and then the assistant will follow those rules automatically for the duration of your project.

We ship one with this handbook, templates/AGENTS.md. It:

  • Tells the assistant that the BuckAI Open-Source Handbook is the authoritative reference.
  • Links every step (repo hygiene, Zenodo DOI, PyPI, Docker, paper.md, JOSS form) to the specific chapter.
  • Bakes in the “MUST NOT” list (Co-Authored-By trailers, marketing language, unauthorised force-pushes, pasted credentials, fabricated APIs).
  • Bakes in the “SHOULD” list (ask before destructive operations, explain trade-offs, verify names, use inline math for Unicode, commit-message quality).
  • Has a customisation section at the bottom for project-specific notes.

How to install it

From the root of your project repo (not the handbook repo):

# 1. Download the template
curl -O https://raw.githubusercontent.com/buckai-observatory/buckai-open-source-handbook/main/templates/AGENTS.md

# 2. Give your specific assistant the filename it expects
cp AGENTS.md CLAUDE.md            # Claude Code
mkdir -p .cursor/rules && cp AGENTS.md .cursor/rules/00-open-source-handbook.md   # Cursor
cp AGENTS.md .windsurfrules       # Windsurf
# OpenAI Codex CLI reads AGENTS.md natively — nothing to do
# Gemini CLI: cp AGENTS.md GEMINI.md

# 3. Edit the bottom section of the file to add your project-specific state
# (author list, ORCIDs, data-access notes, domain conventions, etc.)

# 4. Commit both AGENTS.md and the tool-specific rename to your repo
git add AGENTS.md CLAUDE.md
git commit -m "Add AI-assistant instruction file from BuckAI Open-Source Handbook"

Now open your project in your favourite agentic assistant. From the first message, it knows the rules, knows where to look up the handbook chapters, and knows not to attribute itself as a co-author on your commits. You can then focus on the substance: “let’s set up pyproject.toml” or “let’s draft paper.md” — and skip re-teaching etiquette every session.

NoteThe template is deliberately conservative

AGENTS.md errs on the side of “ask the user before doing anything with real consequences.” If you find your assistant is being too deferential — asking permission for every trivial edit — trim the MUST-ASK-FIRST list in your project’s copy of the file. What ships in the template is the safe default; your project can loosen it.

Checklist