How to solve Github import error: “We found an error during import”

Hi, i would like to say thank to Github!

From the start of this year (2019-01-07) Github offering us “Free account with unlimited private repositories”. This is really, really good news!

Github was the most popular way to build, contribute and share software. As a developer, beside Stackoverflow, you may spend so much time on Github. We usually staying on Github for reading software document, learn by reading other source code, or fire issues when we have problem…

These benefits is the reason why i love Github and it is a very important part in my development life. It’s about 4 years, the first time i try to put my project source code to Github. But at that time, the only thing  Github offer for Free account is Public Repository. This is not a problem if this was an open source project, but what happen if i want to put my Client’s source code here? I gave up and move to Bitbucket!

Bitbucket make me happy when working with private repository, the only bad thing is the limitation of team member (it’s okay because at that time my team have just 3 peoples)

The next problem that i have to face on Bitbucket: my projects always depend on so many open source, and the most of them are landed on Github. Sometime i have issues with them, or want to contribute some thing, i have to switch to Github… Switching between Github and Bitbucket make me confused.

Sometime later,  my team grow up and my solution was paying Github to become premium 🙂

When the first time i got the good news from Github (https://github.blog/2019-01-07-new-year-new-github/), i decided to move all old project from Bitbucket to Github. Almost projects was imported successful with Github importer; these other projects have error occurred and i can’t move them to Github. Did you ever face this error when importing project to Github?

We found an error during import
We recommend restarting the import process, but if you are still having trouble contact support.

it look like this:

When importing to Github, every thing is okay until this step:

Optimizing repository and pushing commits to Github...

This step keep stucking and never completed! Finally, it throw importing error.

All of my error project was procedure the same behavior. I tried to restart so many time but always failed. The displayed error message was too generally, i just can’t debug or finding the true reason.

Trying to clone the repository into my local computer then push to Github. It was failed too, but now it show me the error message: fatal: pack exceeds maximum allowed

remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 00e28c536177c44a0d5eae**********
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File your-lar-file is 181.94 MB; this exceeds GitHub's file size limit of 100.00 MB

Checking Github file size limit, they won’t allow pushing files with size larger than 100MB (https://help.github.com/articles/conditions-for-large-files/) and all my error projects have.

I removed the large files and trying to re-import. But it still failed with the same error message. Keep calm and i finding that i was doing it too fast. I forgot that when deleting large file, i just deleted it from the last commit; it still existed in older commits and commits history!

I was googling and met BFG Repo-Cleaner, as their description:

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala

BFG Repo-Cleaner will help me removing large file in whole commits history, so cool 😀

Using BFG Repo-Cleaner to remove git’s large files

You should backup whole source code and git database before apply any change bellow!

Java is required to run BFG Repo-Cleaner. Install java first then download BFG from their homepage: https://rtyley.github.io/bfg-repo-cleaner/, for example: bfg-1.13.0.jar

Clone your repository using --mirror flag

git clone --mirror git://git-provider.com/your-repo.git

If just do git clone, you will get only project source code
--mirror flag allows downloading full project Git database, we need this for BFG to work

When the clone process is completed, running the following command (modify “100MB” to your desire value)

java -jar bfg-1.13.0.jar --no-blob-protection --strip-blobs-bigger-than 100M your-repo.git

• --no-blob-protection : allow modify protected commits, if not specified, git will throw error These are your protected commits, and so their contents will NOT be altered
• --strip-blobs-bigger-than 100MB: Remove files with size larger than 100MB

After running above command, change your terminal to project dir. Note that when using --mirror flag, the cloned directory will be named as your-repo.git, not your-repo, run these following commands:

git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push

Now you will able to use Github importer tool.

Conclusion:

Almost problem with Github Importer tool is happen at Pushing commits step; so if you facing it, you should clone and push the source code to Github, git cli will show you the error. Based on that error, you will have detail for finding the solution!

If you are importing from other source, you can import your project to Bitbucket or Gitlab because they don’t limit the file size, then you start over with this tutorial!

Thank you for reading!