Azure DevOps Blog

it seems like it was made at microsoft, using .net[0] I agree it seems quite strange that it's not a git subcommand now that it seems like it's been incorporated into the git core release. I'm not exactly sure of it's utility...seems kinda heavy weight to just be setting recommended configurations? I guess it does some background automation for fetching and gc? [0] https://devblogs.microsoft.com/devops/introducing-scalar/

Pretty cool. Related: I feel like there was a recent post about Scalar by Microsoft, but can’t find it. It’s an alternative to the technique described in the submission. :) https://github.com/microsoft/scalar - “A set of tools and extensions for Git to allow very large monorepos to run on Git without a virtualization layer.” The README has a great summary right at the top. Blog post announcing Scalar: https://devblogs.microsoft.com/devops/introducing-scalar/

Not even the majority of users commenting on this post seem to like this change. This one is my personal favourite response: [0] Makes you wonder if this whole idea of renaming 'master' which in the context of Git has nothing to do with 'master-slave' links makes any sense. I'm still waiting for Mastercard to rename their deeply 'offensive' company name then. /s [0] https://devblogs.microsoft.com/devops/azure-repos-default-branch-name/…

I've used this 'suggestion' workflow in azure devops. https://devblogs.microsoft.com/devops/introducing-the-new-pull-request…

For sure, but most YAML you actually encounter does not use much in the way of JSON syntax, it looks a lot more like this: https://devblogs.microsoft.com/devops/wp-content/uploads/sites/6/2020/… Where arrays and objects just look too similar (IMO), white space is significant, most strings are unquoted, etc. And personally I find it quite difficult to really understand what’s going on there, at a glance, compared to JSON (or JSON5).

That already exists. It’s called Scalar[1] and it has been built-into Git since October 2022[2], dates back to 2020[3] and is the spiritual successor or something Microsoft was using as far back as 2017[4]. 1. https://git-scm.com/docs/scalar 2. https://github.blog/2022-10-13-the-story-of-scalar/ 3. https://devblogs.microsoft.com/devops/introducing-scalar/ 4. https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-plan…

Here's a nice article from the VSTS team at Microsoft on trunk-based development - https://devblogs.microsoft.com/devops/release-flow-how-we-do-branching… This is basically what I do on all of my projects and have put this into place in a couple of companies that I have worked at. Haven't encountered any serious issues and it has offloaded some big problems related to the gitflow madness that preceded it.

It was replaced by Scalar and is now merged into Git: Introduced: https://devblogs.microsoft.com/devops/introducing-scalar/ Integration into Git: https://github.blog/2022-10-13-the-story-of-scalar/

This isn’t true. https://devblogs.microsoft.com/devops/cloud-hosted-mac-agents-for-ci-c… > Please be aware that during this preview, our Mac hardware is hosted in third party datacenters in the United States and your build and release data could cross geopolitical lines. After each build completes, its macOS VM is reimaged, leaving no trace of your data on the agent. For more information, see where VSTS data is stored. Our Mac datacenters will expand to other geographies soon

I expected it to be closer to Microsoft's GVFS: https://devblogs.microsoft.com/devops/announcing-gvfs-git-virtual-file… There, they used a filesystem interface to speed up checkouts for large repositories.

Sounds a lot like Microsoft's Scalar. [1] The commits referenced in the blog post look to be from the Scalar team lead as well. Looks like some of their work is getting upstreamed which is exciting. [2] Although, I wonder what this means for the Scalar project itself. [1] https://devblogs.microsoft.com/devops/introducing-scalar/ [2] e.g., https://github.com/git/git/compare/26bb5437f6defed72996b6a2bb1ff9121ec…

Microsoft/windows is hosted on Azure DevOps, and they have also blogged about what they've done to improve its performance! Here's a recent post: https://devblogs.microsoft.com/devops/introducing-scalar/

Maybe it has something to do with us using the macOS hosts/agents? For deprecating things, I know that they sometimes do brownouts, see https://devblogs.microsoft.com/devops/removing-older-images-in-azure-p…

Yes, I've seen other instances of Microsoft/GitHub making substantial Git contributions: https://devblogs.microsoft.com/devops/supercharging-the-git-commit-gra… Microsoft seemed to have been adopting Git already, so it may be that acquiring Github was merely part of a broader Git strategy that Microsoft has adopted.

https://twitter.com/anoff_io/status/1159512381728407552 True? microsoft person on stage https://devblogs.microsoft.com/devops/author/jeremy-eplingoutlook-com/ Sorry Github! Not good!

In fact there is no stable caching (it's being worked on) and that is a major problem with the system. [1] What pushed us over the edge to take the plunge was out of the box support for MacOS builds. [1] https://devblogs.microsoft.com/devops/adding-caching-to-azure-pipeline…

This may be of interest to you: https://devblogs.microsoft.com/devops/cloud-hosted-mac-agents-for-ci-c…

And on the other end of the scale, this one is a cool real-world example: https://devblogs.microsoft.com/devops/super-charging-the-git-commit-gr…

Some examples, in no particular order. Hampering the productivity: - Review messages get sent out before review is actually finished. It should be sent out only once the reviewer has finished the work. - Code reviews are implemented in a terrible way compared to GitHub or GitLab. - Re-requesting a review once you did implemented proposed changes? Takes a single click on GitHub, but can not be done in Azure DevOps. I need to e.g. send a Slack message to the reviewer or remove and re-add them as reviewer. - Knowing to what line of code a reviewer was giving feedback to? Not possible after the PR got updated, because the feedback of the reviewer sticks to the original line number, which might now contain something entirely different. - Reviewing the commit messages in a PR takes way too many clicks. This causes people to not review the commit messages, letting bad commit messages pass and thus making it harder for future developers trying to figure out why something got implemented the way it did. Examples: - Too many clicks to review a commit message: PR -> Commits -> Commit -> Details - Comments on a specific commit does not shown in the commits PR - Unreliable servers. E.g. "remote: TF401035: The object '<snip>' does not exist.\nfatal: the remote end hung up unexpectedly" happens too often on git fetch. Usually works on a 2nd try. - Interprets IPv6 addresses in commit messages as emoji. E.g. fc00::6:100:0:0 becomes fc00::60:0. - Can not cancel a stage…

> Are there some fundamental aspects of git that would make it either very difficult to improve that, or that would sacrifice some important benefits if they were made? I can't speak to improving git, but I think some light on this area can be shed by Linus' tech talk at Google in 2007. 1. Linus says there's a specific focus on full history and content, not files ... so it's a deliberate, different axis of focus than file count: https://youtu.be/4XpnKHJAok8?t=2586 ... AND it's a specific pitfall to avoid when using Git: https://youtu.be/4XpnKHJAok8?t=4047 2. As Linus tells it, Git appears to be designed specifically for project maintenance while not getting in the way of individual commits and collaboration. But the global history and more expensive operations on things like "who touched this line" are deliberate so lines of a function are tracked across all moves of the content itself. Maintainer tool enablement: https://youtu.be/4XpnKHJAok8?t=3815 Content tracking slower than file-based "who touched this": https://youtu.be/4XpnKHJAok8?t=4071 === I have no answer, but ... Practically, I've used lazy filesystems both for Windows-on-Git via GVFS [1][2] and Google's monorepo jacked into a mercurial client (I think that's what it is?). Both companies have made this work, but as Linus says, a lot of the stuff just doesn't work well with either system. Windows-on-Git still takes a lot of time overall, and stacking > 10 patches of an exploratory refactor with the…

Thanks for the link to the documentation. That is updated with every major Git version, and can be used to track what features are present. Also, the release notes can be helpful. In particular, the recent Git v2.27.0 release does include an implementation of Bloom filters with speedups for `git log` and `git blame`. You need to manually run the command to make it work. The version I prefer is this: > git commit-graph write --changed-paths --reachable After that first write (which writes filters for every reachable commit) you can do a smaller write by adding `--split` to write incrementally [2] [2] https://devblogs.microsoft.com/devops/updates-to-the-git-commit-graph-… By writing these filters, you will speed up most `git log` and `git blame` calls. There is an improvement coming in the next version that includes speedups for `git log -L`. Caveat: The biggest reason these improvements have not been widely advertised is that the user experience has not been completely smoothed out. In particular, you can only write the changed-path Bloom filters using the command(s) above. If a commit-graph is written during GC (due to `gc.writeCommitGraph` config setting) then the filters will disappear. Similar for `fetch.writeCommitGraph`. We plan to have these resolved in time for v2.28.0, along with more performance improvements. (Full disclosure: I am a contributor to Git, Scalar, and VFS for Git, which are referenced by the article.)

That’s quite a few assumptions, at least one of which likely isn’t correct; at Microsoft, the .git directory doesn’t contain ”all the git files, including file versions […] and branche” https://devblogs.microsoft.com/devops/optimizing-git-beyond-gvfs/: ”GVFS allows our developers to simply not download most of those 3.5 million files during a clone, and instead simply page in the (comparatively) small portion of the source tree that a developer is working with.” Having said that, the linked https://docs.microsoft.com/en-us/azure/devops/learn/git/gvfs-design-hi… claims Windows gives a size about half that 500GB for the Windows source code: ”the Windows repo, which has over 3 million files in the working directory, totalling 270GB of source files. That’s 270GB in the working directory, at the tip of master. To clone this repo, you would have to download a packfile that is about 100GB in size, which would take hours. And once you’ve succeeded in cloning it, local git operations like checkout (3 hours), status (8 minutes), and commit (30 minutes) would still take way too long to run”.

Microsoft had a bunch of solutions to handle their massive Windows repo: VFS for Git (GVFS), Scalar, and now it has a bunch of MS specific patches on top of the official git client, but apparently that one is also not required any more as partial clone is now supported on azure as well (which is another such implementation from Microsoft employees that made it to both GitHub and upstream git). So yeah, solved problem thanks to Microsoft. Solved multiple times in fact, and because it's Microsoft, only not all had Linux installers: https://github.com/microsoft/scalar/issues/323#issuecomment-927865734 https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-s… https://devblogs.microsoft.com/devops/introducing-scalar/ https://github.com/microsoft/git https://devblogs.microsoft.com/devops/git-partial-clone-now-supported-…

microsoft: in 2017, the Windows git repo contained ~3.5 million files & used 300GB when checked out as a git repo [1], In 2020, the Office git repo contained ~3 million files [2]. facebook: In 2011, the main site had around 9.2 million lines of code "excluding numerous backend services" [3]. In 2014 the android codebase was 4 million lines of code and the main site without backend code was 62 million lines of code [3]. In 2014, when facebook were talking about scaling mercurial, the main source repo was described as "many times larger than even the Linux kernel, which checked in at 17 million lines of code and 44,000 files in 2013". Around 2018 (?) facebook started using "eden", a fork of mercurial [5]. If we make the working assumption that expected lines-of-code / file of 2013 linux kernel is representative of all repos, then that gives around 386 lines-of-code / file. loc(microsoft) >= loc(windows in 2017) + loc(office in 2020) ≈ 386 loc / file * (3.5m + 3m) ≈ 2.5 billion lines of code The linked blog posts from microsoft & facebook talking about scaling version control systems are fairly interesting! [1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-plan… [2] https://devblogs.microsoft.com/devops/introducing-scalar/ [3] https://www.quora.com/How-many-lines-of-code-is-Facebook [4] https://engineering.fb.com/core-data/scaling-mercurial-at-facebook/ [5] https://github.com/facebookexperimental/eden