Brian Harry

Quoting Brian Harry from a comment response at https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep… "It depends a great deal on the operation. SourceDepot was much faster at some things – like “sd opened”, the equivalent of “git status”. sd opened was < .5s. git status is at 2.6s now. But SD was much slower at some other things – like branching. Creating a branch in SD would take hours. In Git, it's less than a minute. I saw a mail from one of our engineers at one point saying they'd been putting off doing a big refactoring for 9 months because the branch mechanics in SD would have been so cumbersome and after the switch to Git they were able to get the whole refactoring done in a topic branch in no time. On an operation, by operation basis, SD is still much faster than our Git/GVFS solution. We're still working on it to close the gap but I'm not sure it will ever get as fast at everything. The broader question, though is about overall developer productivity and we think we are on a path to winning that."

> It has nothing to do with the Windows filesystem; Git simply cannot support a 5 GB working tree on any filesystem. Can you provide a reference? I was searching a bit and only things I found was bugs in windows[1] for git lfs. > You can call this "pathological" but this throws a lot of shade on monorepos without much critical examination of how or when they might be useful. Windows codebase has 3.5 million files and its repo is 300GB in size. It is not normal. This is google or MS type of problem and not average git user. MS instead changing workflow decided to create GVFS[2] [1] https://github.com/git-lfs/git-lfs/issues/2434 [2] https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Well even the entire windows sits in a single git repo https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Unless you change git like Microsoft did: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Microsoft went from multiple smaller repos for windows to one large one: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

was this when they decided, nah fk it let s just aquire the thing https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

I guess now this makes sense: https://blogs.msdn.microsoft.com/bharry/2017/03/31/shutting-down-codep…

Note that the actual link is https://blogs.msdn.microsoft.com/vsoservice/?p=16295 but I am hesitant to call out the current link at https://blogs.msdn.microsoft.com/bharry/2018/03/02/a-good-incident-pos… as blogspam.

Uh, they moved all of Windows into a single git-repo. It used to sprawl over 40+ Source Depot repos but now it's one gigantic monorepo [1]. [1] https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som…

1. Git LFS is not part of Git. It's an unofficial extension, despite the name. I've got my own qualms about Git, but that's like complaining about GitHub's forking model as a reason Git is bad. 2. Microsoft uses Git in single large-scale mono repo form, for - believe it or not - Windows. It clocks in at about 3.5 million files, and a full clone weighs in at 300GB: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Microsoft has released some tools to support that for Git. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep… https://blogs.msdn.microsoft.com/devops/2017/05/30/optimizing-git-beyo…

> monorepo this size would simply not scale on git https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

I've been thinking of it as more of making it on-demand — it doesn't sound like you lose the underlying distributed nature (e.g. from the description on https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som… it sounds like 'git push ssh://github.com/microsoft/windows.git' would work, although it might take longer to run than it does for you to be fired) but managed clients can choose to operate in a mode where they download everything on-demand rather than immediately. That seems like a reasonable compromise for a workload which Git is otherwise completely unable to handle.

I think once you're checked out, you're still distributed as far as being able to make and push commits which modify the files you have. But obviously someone can't clone files from you which you don't yourself have. See previous articles which discussions: https://blogs.msdn.microsoft.com/devops/2017/02/03/announcing-gvfs-git… https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Visual Studio Team Services. I'm not sure what you mean by "massive" but VSTS is the only service that handles the truly massive projects, like Microsoft Windows. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep… Edited to add ObDisclaimer: I work on the VSTS team.

I work on the Windows team. Things are very different now. Source control especially: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep… It still takes a lot of political effort to change primary UI features though. And I think that's a good thing.

they have definitely changed the source control infrastructure and are now on git. https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

As does Microsoft: https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

Here is the blog post right from MS on this... https://blogs.msdn.microsoft.com/bharry/2017/05/24/the-largest-git-rep…

This was discussed a bit in the comments in Brian Harry's last post on GVFS: https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som… We're building a VFS (Virtual File System) for Git (G) so GVFS was a very natural name and it just kind of stuck once we came up with it.

Here is Brian Harry at MS on this very topic about a year ago: https://blogs.msdn.microsoft.com/bharry/2015/03/12/the-future-of-team-… > "Most of our customers still use TFVC and we value this tremendously. Most people in Microsoft still use TFVC. Most new projects created today on VS Online choose TFVC." Etc. It's quick and to the point, go read it. And again, has there been some sea-change at MS over the last year on source control? Quite possibly. But so far you've offered nothing but your opinion.

Parts of Microsoft actually do respond and provide nice roadmaps. I am hoping Github will do something similiar. https://blogs.msdn.microsoft.com/bharry/2016/01/11/team-foundation-ser…

Scalability to large numbers of files and large numbers of changesets. For the large number of files one https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-f… is one source. There's some earlier discussion of the same issue at https://news.ycombinator.com/item?id=3549679 that goes into some technical details. There has been some work on git since then to address some of those issues (e.g. see https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som… ) but it's not clear to me that it helped enough to catch up to where Mercurial is for large repos. For large numbers of changesets, just try running "log" or "annnotate" on any file with a long history in git. I just did this simple experiment: 1) hg clone https://hg.mozilla.org/mozilla-central/ 2) git clone https://github.com/mozilla/gecko-dev.git 3) (cd mozilla-central && time hg log dom/base/nsDocument.cpp) 4) (cd gecko-dev && time git log dom/base/nsDocument.cpp) It's not quite apples to apples because the git repo there has some pre-mercurial CVS history in it. But note that I'm not even using --follow for git and the file _has_ been renamed after the mercurial repo starts, so git is actually finding fewer commits than mercurial is here. Anyway, if I do the above log calls a few times to make sure the caches are warm, I end up seeing times in the 8s range for git and the 0.8s range (yes, 10x faster) for mercurial. That all said, most repos do not have millions…

I can't see it being easy to take a 20 year massive code base and split it up into components that are so isolated enough they can live in different repositories smoothly. As they say in: https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som… The first big debate was – how many repos do you have – one for the whole company at one extreme or one for each small component? A big spectrum. Git is proven to work extremely well for a very large number of modest repos so we spent a bunch of time exploring what it would take to factor our large codebases into lots of tenable repos. Hmm. Ever worked in a huge code base for 20 years? Ever tried to go back afterwards and decompose it into small repos? You can guess what we discovered. The code is very hard to decompose. The cost would be very high. The risk from that level of churn would be enormous. And, we really do have scenarios where a single engineer needs to make sweeping changes across a very large swath of code. Trying to coordinate that across hundreds of repos would be very problematic.

their earlier blogpost goes into it a bit https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-som… : The first big debate was – how many repos do you have – one for the whole company at one extreme or one for each small component? A big spectrum. Git is proven to work extremely well for a very large number of modest repos so we spent a bunch of time exploring what it would take to factor our large codebases into lots of tenable repos. Hmm. Ever worked in a huge code base for 20 years? Ever tried to go back afterwards and decompose it into small repos? You can guess what we discovered. The code is very hard to decompose. The cost would be very high. The risk from that level of churn would be enormous. And, we really do have scenarios where a single engineer needs to make sweeping changes across a very large swath of code. Trying to coordinate that across hundreds of repos would be very problematic. After much hand wringing we decided our strategy needed to be “the right number of repos based on the character of the code”. Some code is separable (like microservices) and is ideal for isolated repos. Some code is not (like Windows core) and needs to be treated like a single repo.

This is a bit alarmist. If github went away or became like Sourceforge of old then we could just move to something else or do self-hosting again. Lose a chunk of the internet by moving popular software like Handbrake to GitHub? how could that happen? Most multiplatform open source software like this have copies of their source code across thousands of linux distribution mirrors. Also, google code never went away. It just stopped working as an active platform, but Google still keeps the archive of what already existed there, to this day: https://code.google.com/archive/ Microsoft is doing the same after shutting down their Code Plex because of moving toward GitHub: https://blogs.msdn.microsoft.com/bharry/2017/03/31/shutting-down-codep… > At that time, CodePlex.com will start serving a read-only lightweight archive that will allow you to browse through all published projects – their source code, downloads, documentation, license, and issues – as they looked when CodePlex went read-only. You’ll also be able to download an archive file with your project contents, all in common, transferrable formats like Markdown and JSON. Where possible, we’ll put in place redirects so that existing URLs work, or at least redirect you to the project’s new homepage on the archive. And, the archive will respect your “I’ve moved” setting, if you used it, to direct users to the current home of your project. If there is anything to lose after GitHub's shutdown at some distant point in the…