Copying a Git repository with full commit history

Contents

• Introduction
• The script
• How it works
• Usage
• Things to keep in mind

Introduction

I recently needed to migrate a Git repository into another repository while preserving the complete commit history, authors, and timestamps. This comes up when consolidating multiple projects or moving a repo into a monorepo structure.

I found a solution in this PR by Blake Friedman. The approach uses a bash script to replay commits from one repository into another. Here is a slightly modified version of this script. Feel free to use AI to modify it to your use case.

The script

#!/bin/bash
 
set -e
 
# Configuration - Update these paths to match your setup
SOURCE_REPO_PATH=~/workspace/source-repo
TARGET_REPO_PATH=~/workspace/target-repo
TARGET_SUBDIRECTORY=native
 
COMMITS=/tmp/commits
MESSAGE=/tmp/message
AUTHOR=/tmp/author
DATE=/tmp/date
 
CURRENT=head
 
TOTAL=$(wc -l $COMMITS | awk '{ print $1 }')
 
i=1
echo "Starting"
while true; do
  if [[ -s $COMMITS ]]; then
      echo "Grabbing next commit: $i/$TOTAL"
      COMMIT=$(head -n 1 $COMMITS)
      tail -n +2 "$COMMITS" > "${COMMITS}.tmp" && mv "${COMMITS}.tmp" "$COMMITS"
  else
      echo "DONE: All commits done"
      exit 0
  fi
  pushd "$SOURCE_REPO_PATH"
  git checkout --force $COMMIT
 
  export GIT_AUTHOR_NAME=$(git log --format=%an -n 1 "$COMMIT")
  export GIT_AUTHOR_EMAIL=$(git log --format=%ae -n 1 "$COMMIT")
  export GIT_AUTHOR_DATE=$(git log --format=%ad -n 1 "$COMMIT")
  export GIT_COMMITTER_NAME=$(git log --format=%cn -n 1 "$COMMIT")
  export GIT_COMMITTER_EMAIL=$(git log --format=%ce -n 1 "$COMMIT")
  export GIT_COMMITTER_DATE=$(git log --format=%cd -n 1 "$COMMIT")
 
  git log --format=%B -n 1 "$COMMIT" > $MESSAGE
  echo "Original: https://github.com/organization/source-repo/commit/$COMMIT" >> $MESSAGE
  rm -rf "$TARGET_REPO_PATH/$TARGET_SUBDIRECTORY"
  rsync -av --exclude='.git' ./ "$TARGET_REPO_PATH/$TARGET_SUBDIRECTORY/"
  popd
 
  git add "$TARGET_SUBDIRECTORY/"
  git commit -F $MESSAGE --no-verify || echo "No changes to commit, continuing..."
 
  echo "Progress: $i/$TOTAL"
  sleep 0.3
  let i=i+1
done

Note: this version of the script assumes you are copying whole source repo to a folder in the target repo.

How it works

First, generate a list of commits in chronological order:

git log --format=%H --reverse > /tmp/commits

The script reads commits one at a time and for each commit:

1. Checks out the commit in the source repository
2. Extracts author, committer, and date information
3. Copies files (excluding .git) to the target repository subdirectory using rsync
4. Creates a new commit with the original metadata and appends a link to the original commit

Git uses environment variables like GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, etc. to determine commit metadata, which is how the script preserves the original information.

Usage

Set up your repositories:

git clone https://github.com/organization/source-repo ~/workspace/source-repo
git clone https://github.com/organization/target-repo ~/workspace/target-repo
cd ~/workspace/target-repo

Generate the commits list:

cd ~/workspace/source-repo
git log --format=%H --reverse > /tmp/commits

Update the configuration variables at the top of the script to match your setup:

• SOURCE_REPO_PATH: Path to your source repository
• TARGET_REPO_PATH: Path to your target repository
• TARGET_SUBDIRECTORY: Subdirectory name in the target repo (e.g., native)

Then run:

chmod +x migrate.sh
./migrate.sh

Things to keep in mind

• Backup everything before running this script
• The script uses --force checkout, which discards local changes
• The sleep 0.3 delay allows time for git lock files to be deleted between commits
• Empty commits are skipped gracefully
• Merge commits are replayed as regular commits

And that's it! I hope you found this article useful. If you have any questions or feedback feel free to reach out to me on Twitter.