Git LFS for Large Files

By Khimananda Oli | Last reviewed: August 2026

Standard Git repositories degrade rapidly when you commit binaries like datasets, compiled libraries, or high-resolution media directly into history. Implementing Git LFS for large files solves this by replacing heavy objects with lightweight text pointers while storing the actual content on a separate remote server. This approach keeps clone times fast and history manageable without forcing developers to abandon their familiar Git workflow.

How do you correctly configure Git LFS for large files?

Configuration is deceptively simple but requires discipline. A common mistake I see in audits is installing LFS globally but forgetting to verify per-repository hooks, leading to silent failures where multi-gigabyte binaries still enter standard packfiles. Always treat LFS setup as code, not just a local environment preference.

Installation and initialization

Before touching any repository, ensure the client extension is active. On Ubuntu/Debian systems in 2026, the canonical method uses the official APT repository rather than outdated distro packages:

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install

The final command installs post-checkout and pre-push hooks in your global Git config. For shared team environments, document this step in your onboarding wiki or automate it via Ansible playbooks for developer workstations to guarantee consistency across Nepal-based and remote teams.

Defining tracking rules

Never track everything. Be surgical with patterns in .gitattributes to avoid accidentally capturing temporary build artifacts or IDE caches:

# Track ML model weights and datasets
*.pt filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
data/raw/ filter=lfs diff=lfs merge=lfs -text

# Media assets
assets/videos/*.mp4 filter=lfs diff=lfs merge=lfs -text
design//*.psd filter=lfs diff=lfs merge=lfs -text

# Exclude temp files explicitly
!*.tmp
!.cache/

After editing, stage the attributes file immediately: git add .gitattributes. This file must be committed before any LFS-tracked content; otherwise, collaborators cloning the repo will download raw pointers instead of actual files.

How do you migrate an existing repository to Git LFS?

Retrofitting LFS onto a repo that already contains large binaries is the most operationally risky task. Simply adding tracking rules going forward does nothing to shrink existing history. You must rewrite commits, which changes every SHA downstream. Coordinate this carefully with your team and align it with your broader CI/CD pipeline maintenance windows.

Rewriting history safely

Use git lfs migrate in import mode to scan all branches and convert matching files retroactively:

# Dry-run first to inspect what will change
git lfs migrate info --include="*.bin,*.dat" --everything

# Actual migration (irreversible without backup)
git lfs migrate import --include="*.bin,*.dat" --everything \
  --include-ref=refs/heads/main \
  --include-ref=refs/heads/develop

This process can take hours for repositories with years of binary churn. Run it on a fresh clone, never on your primary working copy. After completion, force-push all rewritten branches and require every collaborator to re-clone. There is no safe way to merge rewritten history back into old clones.

Validating migration success

Verify that no large objects remain in standard packfiles:

• git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectsize)' | awk '/^blob/ {print $2}' | sort -rn | head — confirms largest blobs are now small pointers.
• git lfs ls-files — lists all currently tracked LFS objects.
• Clone the migrated repo fresh and time it; expect 70–95% reduction in initial clone duration.

What are the storage limits and costs for Git LFS?

LFS shifts cost from compute/network to dedicated object storage. Pricing models vary dramatically between providers, and exceeding free tiers is the #1 budget surprise I encounter when auditing cloud spend for startups. Always model costs before adoption.

For Nepal-based teams managing sensitive data or operating under strict budget constraints, self-hosting LFS against MinIO or a local S3-compatible gateway often makes more sense than paying recurring SaaS fees. Pair this with cloud cost optimization tactics to keep storage expenses predictable.

When should you avoid Git LFS entirely?

LFS is not a universal solution. In my experience supporting infrastructure across AWS, Azure, and on-prem environments, these scenarios demand alternatives:

• Files changing every commit: LFS stores full copies per version, not deltas. A 500MB dataset updated daily creates massive storage bloat. Use DVC, LakeFS, or direct S3 versioning instead.
• Public open-source projects: Most free-tier LFS quotas are insufficient for popular repos. Host releases on GitHub Releases, SourceForge, or Cloudflare R2 with signed URLs.
• Build artifacts: Compiled binaries belong in artifact repositories (Nexus, Artifactory, ECR), not source control. See container registry guides for proper separation.
• Sub-second access latency required: LFS adds HTTP round-trips during checkout. If your CI pipeline checks out thousands of small LFS files, consider vendoring or submodule strategies.

Performance pitfalls to watch

Even when LFS is appropriate, misconfiguration causes pain. The lfs.fetchexclude and lfs.fetchinclude configs are essential for monorepos where developers only need subsets of tracked assets. Without them, every clone pulls terabytes unnecessarily. Also enable lfs.concurrenttransfers (default 3) to match your network capacity; raising it to 8–12 significantly improves throughput on high-bandwidth connections common in Kathmandu's newer fiber deployments.

How do you troubleshoot common Git LFS failures?

Most LFS issues stem from hook misalignment or authentication expiry. Start diagnostics here:

1. Pointers downloaded instead of files: Run git lfs pull. If it fails, check git lfs env for endpoint mismatches. Verify .gitattributes was committed before the affected files.
2. Push rejected with "batch response": Usually auth or quota. Test with GIT_TRACE=1 GIT_CURL_VERBOSE=1 git push to inspect HTTP responses. Rotate tokens if using PATs; SSH-based LFS auth is more reliable long-term.
3. Smudge filter errors on checkout: Corrupt cache. Clear with git lfs prune then git lfs fetch --all. Check disk space; LFS cache defaults to ~/.local/share/lfs and can exhaust small VPS volumes.
4. CI timeouts: Add GIT_LFS_SKIP_SMUDGE=1 to skip downloading during shallow clones, then fetch only needed paths explicitly. This alone has rescued dozens of failing pipelines I've audited.

Implementing Git LFS for Large Files With Confidence

Adopting Git LFS for large files is straightforward mechanically but demands thoughtful governance around cost, migration risk, and team workflows. Start with clear tracking policies, validate migrations on isolated clones, monitor storage spend monthly, and maintain runbooks for the inevitable auth or cache issues. When implemented with this level of rigor, LFS preserves Git's collaborative strengths without sacrificing performance at scale.

If your team needs help designing LFS architecture, migrating legacy repositories safely, or integrating LFS with compliant cloud storage backends, reach out to discuss your infrastructure requirements. I regularly assist organizations across Nepal and globally with version control strategies that balance developer velocity with operational sustainability.