Pivotal Blog

I had been quite skeptical of Clean Architecture when I first came across it. I don't find Uncle Bob's post on it particularly insightful; for me it's not vocational enough. Then a few years ago I had to maintain a software stack written by contractors from Pivotal (https://pivotal.io/) in a Clean Architecture style - it was truly a revelation for me; akin to that "aha" moment of fully grokking homoiconicity in LISPs. Now, up until this point, all code I'd encountered at Twitch heavily reflected the domain of the problem it was solving and the technology in which it was written. That is, if you were looking at a certain piece of architecture you had to really fully understand not only exactly the intent of that code base but also the details of the framework in which it was written. As codebases increased in number and size, it became much harder to scale as an engineer. Jumping from a Rails monolith, to a highly concurrent Golang HTTP CRUD-API, to a highly asynchronous Twisted-Python request routing system (broadly the main three backend techs) had a very large cognitive load. The eng org cleaved along these lines and maintaining velocity in that world was very hard; attempts to introduce new tech or join chunks of the org took on a "religious" tone. So initially coming across this clean architecture stack felt very similar to that. It had a lot of "weird new things" in it, but once I understood that much of it was routing (tagging inbound requests in a manner that a…

> promote an unrealistic image of the "Software Developer Career" ... and within 6 months ( seriously ? ) you'll be ready and making 120K$/Year. Huh? What's "unrealistic" about it? Lambda and App Academy regularly graduate students making that kind of money. > most of the stuff a developer has to do has often nothing to do with the technology he learnt at school Given you agree that much of what you learn during a CS degree gets supplanted by real-world knowledge once you get a job, I don't see what your problem is with bootcamps like this one. (Makers Academy grad here; 4 years into tech and gainfully employed at http://pivotal.io thank you very much)

I get downvoted a lot of the time if I mention Pivotal or any of the things we work on (particularly Cloud Foundry). However in this case it might well be the referral link. Some people don't like them on principle and there will always be someone who hasn't seen previous discussions about it[0]. For anyone who hates referral links, go to https://pivotal.io/careers. If I have to miss out on a referral in exchange for a good hire, it's worth it in the long run. [0] Notably, this one: https://news.ycombinator.com/item?id=12202874

This is a 2 year old article. Where's the follow up? What did they end up doing? EDIT: AHA! Part 2: https://blog.pivotal.io/labs/labs/ntp-server-costing-500year-part-2-ch…

I'm not sure how much I can help because I didn't setup the RMQ cluster so I don't know the configuration details but I know it is fairly powerful (its also partly why I can't entirely be critical of Auth0 because ZMQ is probably far less expensive infrastructure wise). I do know we use multiple queues and even exchanges (and I did not know about the one core to queue). A simple googling shows though folks have achieved far greater throughput[1] than 100k (and by the way this wasn't sustained.. it was spikes). [1]: https://blog.pivotal.io/pivotal/products/rabbitmq-hits-one-million-mes…

I'm sorry I'm not aware of the person, but would be interested in reading/seeing it. A quick google search yielded https://blog.pivotal.io/pivotal-perspectives/features/revisiting-micro… - Is this the one you referred? Btw - the list is not an invention worth copying. All items in that are not novel or unique, they are general practices that every good engineer would have their own version of. I'd like to see the talk and try to add a few more items to this.

What you are describing is a pretty poor pairing interview process. I did a bunch of really frustrating whiteboard type interviews and sudo pairing interviews like you describe before interviewing for my current position [1]. The screen is a one hour onsite pairing session followed by a final round that is a day of pairing using a proper workstation: One computer, two monitors and two sets of mice/keyboards. You pair with one person in the morning and another in the afternoon. It's very relaxed (you take turns "driving") and it gives you and the interviewers a lot more information about each other and your potential place of work. It also gives you enough time to adjust to the process if you are nervous. I don't think I'd do it any other way going forward. [1] http://pivotal.io/careers

Yes, true. Here are the technical details: http://pivotal.io/big-data/case-study/distributed-in-memory-data-manag… - a worthy HN submission by itself

Feedback? How about just not making wild claims about byzantine fantasy designs that you never tested under any kind of load. There has been a lot of research in messaging architectures, some of the best message brokers are free. As it happens, none of them have any resemblance to your proposed design. RabbitMQ has been benchmarked[1] to 1 million messages/sec on 30 servers and works very well for many people. Why not start with that? [1] http://blog.pivotal.io/pivotal/products/rabbitmq-hits-one-million-mess…

Pairs of RabbitMQ brokers, used in a Publish-One-Subscribe-Many fashion for availability. Scaling RabbitMQ at SoundCloud: http://vimeo.com/69960105 Edit: usually with durable queues on SSDs Edit: text version of the talk: http://blog.pivotal.io/pivotal/case-studies-2/scaling-with-rabbitmq-so…

> They have their own sauce. I personally called it the Pivotal Process What Pivotal does is fairly vanilla Extreme Programming. Indeed, the vanillaness of it is something of a selling point. In the London office, we would start projects by giving client PMs and programmers a copy of Extreme Programming Explained and telling them that was what we were going to do. It's easier to coax them into following a process if it's a documented industry-standard one rather than some special sauce. Pivotal does have a few extra bells and whistles that fill in spaces around the XP core - "discovery and framing" [1] and "inception" [2] being two that spring to mind. And as you say, there is less emphasis on iterations as a unit of work - the planning meeting is weekly, but it feeds a continuous flow of development and delivery. But the daily pattern of work should be familiar to anyone who knows XP. [1] https://content.pivotal.io/blog/what-we-did-on-our-summer-internship-a… [2] https://www.infoq.com/articles/project-inception-meeting/

Would the cost/savings also depend on the 'predictability' of the infrastructure needs? To me, it seems like there is a dependency. For example if I can project steady growth or decline of my needs, it seems that I benefit less from the infrastructure-on-demand like AWS/Azure/GC Also, depending on the size of the organization it seems that there at least several options on data centers: - have my own data center - collocate my hardware in other's data center - rent hardware - use infrastructure-on-demand + software-as-a-service AWS/Azure/GC are in 4th category, but cost savings in terms of electricity/building capacity/peering points are available in the first 3 as well. Many organizations deploy Pivotal's https://pivotal.io/industries/financial-services or Redhat/OpenStack So they are not actually 'building' the infrastructure management layer from scratch. Also for investment banks (due to exchange connectivity) , telecoms (due to peering points, physical tower locations) always going to have their 'Front-office' applications in data centers that offer lowest latency for those apps. Midoffice applications (or BOSS as they are labeled in telecom) can live in data centers reasonably distant (in terms of ping time) from the front office. So the infrastructure for those has a different cost profile. Most of this applications touch trades or trade events that carry millions of dollars worth of contract value (not Facebook's likes or favorites... ) There is…

In my view what we're really talking about here is PaaS. Every shop is left to implement a private PaaS on their own for the most part. There are software companies out there who specialize in helping teams deploy this kind of architecture for example: https://pivotal.io/platform http://deis.io/ both are open source technologies based on Docker that are gathering momentum and you can hire consultants to help you deploy either one. I personally don't use Docker. The startup I'm building has chosen to standardize on the JVM for all application code so we leverage the JAR file as a kind of container. The Java ecosystem already solved the problem of zero-downtime deployments a long time ago so for us deploying can be as simple as shipping new jars file across the network. Instead of using Docker to drive development we simply spin up development database/redis/etc instances in the cloud which automatically join a development VPN network. All of the non-VPN interfaces are automatically firewalled off. One nice advantage of this setup is that developers who have slow laptops are still able to work. I'm a big fan of this approach. Check out Wildfly's "High Availablity" features if you're interested in one way that the Java Ecosystem can make headaches like zero-downtime deployment, HTTP health checks, monitoring, caching, and even load balancing disappear.. It'll deploy non-java code too as long as it's on the JVM. If you're a Scala only shop there are some great…

I wouldn't really try to put a name on it other than Pivotal Process or Pivotal Methodology... I was just googling around to see if they even name it and found these articles... https://infocus.dellemc.com/rahul_bhandari/software-development-dell-a… (corporate version) https://pivotal.io/agile (this one bugs me a bit since it really just compares against waterfall) https://medium.com/@ryanjjones10/what-is-the-pivotal-process-417fab075… (this one is more accurate)

Beyond the 12 factor app: (1) One codebase, one application (2) API first (3) Dependency management (4) Design, build, release, and run (5) Configuration, credentials, and code (6) Logs (7) Disposability (8) Backing services (9) Environment parity (10) Administrative processes (11) Port binding (12) Stateless processes (13) Concurrency (14) Telemetry (15) Authentication and authorization More info: https://content.pivotal.io/blog/beyond-the-twelve-factor-app

I agree, but it's fun and sometimes enlightening to let the debate be framed this way Compare[1] Chapter 1: > “Nobody has ever built cloud-native apps without a platform. They either build one themselves or they use Cloud Foundry.” [1]: https://content.pivotal.io/ebooks/cloud-foundry-the-cloud-native-platf…

Pivotal and VMware are working on this. From what I’ve seen (yet to play around too deep with this yet) looks promising. You’ve got it spot on though, just needs to be simple. https://pivotal.io/containers/deploying

Is your product like Pivotal's PCF / PAS? https://pivotal.io/platform/pivotal-application-service

There seems to be also other companies involved: Redhat: https://blog.openshift.com/red-hat-istio-launch/ Pivotal: https://content.pivotal.io/blog/pivotal-and-istio-advancing-the-ecosys…

part 2: https://blog.pivotal.io/labs/labs/ntp-server-costing-500year-part-2-ch…

Second part is here: https://blog.pivotal.io/labs/labs/ntp-server-costing-500year-part-2-ch…

You should consider getting in touch with the folks at Pivotal (http://pivotal.io/locations/london). They have a very progressive working practice, and are keen to attract diverse staff. They're based in Old Street.

"I'd go so far as to say that I've never heard of a company who got this working before, so if that is the case then Pivotal are clearly doing something right." I hadn't really seen it before either, TBH. It is not perfect - sometimes we ship late. But that's part of the culture - Pivotal Tracker (www.pivotaltracker.com) tells you the date you're expected to ship given the work you have left, you don't tell it when to ship ;) Management won't break people over a deadline, they'll generally slip the date. Sustainability is more important. We do have a small/growing office in Sydney with a few openings: http://pivotal.io/locations/sydney

Also Indian railways: 200,000 Concurrent ticket Purchases https://pivotal.io/big-data/case-study/distributed-in-memory-data-mana…

Didn't understand much from the linked page, but I found this website (from Pivotal, the commercial entity behind Geode) quite informative. Perhaps it's useful to others. https://pivotal.io/big-data/pivotal-gemfire === I found this interesting deployment: China National Railways use Geode to run railway ticketing for the entire country with a 10 node cluster, managing 2 TB of "hot data" in memory, and 10 backup nodes for high availability and elastic scale. Holiday travel periods [Chinese New Year's] create peaks of 15,000 tickets sold per minute, 1.4 billion page views per day and 40,000 visits per second. http://pivotal.io/big-data/case-study/scaling-online-sales-for-the-lar…

This is awesome. What I also found really cool is that this inadvertently lets you "hear the sound" of chrome/firefox throttling the `setTimeout` calls when the tab is inactive [1]. Run one of the sorting algorithms in chrome or firefox and then switch to a new tab. You'll notice it slows down. Switch back and forth between the tabs and it's almost like you are conducting of the algorithm :) [1] http://blog.pivotal.io/labs/labs/chrome-and-firefox-throttle-settimeou…

Blog post about the service: http://blog.pivotal.io/labs/labs/sslip-io-a-valid-ssl-certificate-for-…

According to the Redis support page, Pivotal sponsors Redis and they provide commercial support services for it. Although, to be honest, the page linked from the Redis site (http://www.pivotal.io/big-data/redis) doesn't seem to have much to say on the matter. Looks like an ad for their cloud platform, and I wouldn't have guessed they offer support for Redis itself.

I thought you were wrong but not so. Here's the proof that they are indeed the same company: http://pivotal.io/labs

> As for Java and C# on the web. I just wish it was more accessible to smaller companies. 100% of my freelance stuff are using cheap hosting that rarely support anything more than Perl/PHP. One year ago I would have agreed with you, but I have since discovered some of the newer PaaS solutions like CloudFoundry / Pivotal CF [1] and OpenShift [2]. I'm using Pivotal CF for my current side project (a public website + web service) and am paying about $7/month. [1] http://www.pivotal.io/platform-as-a-service/pivotal-cf [2] https://www.openshift.com/