I have been doing some tech advice jobs on the side to see what's going on in the world and it's really scary what I found. Only yesterday I was talking with the cto of a niche social networking company that has a handful of users and probably won't get much more who was telling me the tech they use; Node, Go, Rust, Mongo, Kafka, some graph db I forgot, Redis, Python, React, Graphql, Cassandra, Blockchain (for their voting mechanism...), some document database I had never heard of and a lot more. A massive, brittle, SLOW (!) bag of microservices and technologies tied together where in 'every micro part' they used best practices as dictated by the big winners (Facebook, Google, whatever) in Medium blogs. It was a freak show for a company of 10 engineers. But this is not the first time I encounter it; 3 weeks ago, on the other side of the world, I found a company with about the same 'stack'.
People really drink the koolaid that is written on these sites and it is extremely detrimental to their companies. PostgreSQL with a nice boring Java/.NET layer would blow this stuff out of the water performance wise (for their actual real life usecase), would be far easier to manage, deploy, find people for etc. I mean; using these stacks is good for my wallet as advisor, but I have no clue why people do it when they are not even close to 1/100000th of Facebook.
Been there. OTOH, my last job was at a startup that used a LAMP stack but made enough money to be self-sufficient and not depend on VC money to keep running.
When the legacy systems started to hurt us (because they were written by the founder in a couple of weeks in the most hacky way), we decided against microservices and went to improve the actual code into something more performing and more maintenable,also moving from PHP5 to PHP7.
As much as we all wanted to go microservices and follow the buzz, we were rational enough to see that it didn't make any sense in our case.
Resume driven development. I've worked for a similar small company with barely any users/data but the tech choices were driven by how useful is the tech for their future job prospects and not the current or future needs of an organization.
Maybe if companies gave sufficient raises and promotions, and actually tried to retain talent, then we wouldn't have this culture where people keep having to switch jobs, and therefore always be looking out for what will get them the next gig.
I think engineers would still jump ship just as often even if they were paid more. When you really get down to it, most programming is pretty tedious. What makes it fun, for some engineers, is the opportunity to learn new things, even it means doing so at the detriment of the business.
I had a similar experience last year as an advisor to a large "enterprise" type company. They are a slow-moving company with no competitive pressure & tons of compliance constraints, had several teams of Java engineers, they all know Java really well, they do not know JavaScript well. They had been attempting to migrate all their services to a modern, cool, bells & whistles JS SPA architecture.
Get ready for the surprise twist: It wasn't going well. I was hired as an expert JS consultant to advise them on which JS framework to use.
My advice? Get ready for surprise #2: "don't use javascript [or use it sparingly as needed]."
Look, for most websites renting less than a handful of dedicated boxes would be enough for all purposes -- dev, staging, production. Instead they rack up an astonishing cloud bill from Amazon. It's not hype. It's boring. But it goddamn works. Sometimes I feel I am the last warrior of the YAGNI tribe.
You are not. I am only 38 and I only work professionally as a programmer for 16.5 years but I am already quite conservative in terms of ruthlessly eliminating complexity and removing any tech that can be consolidated in a monolith repo without huge amounts of effort.
Experience helps a lot. When you know that complexity and too much diversity breeds tech debt, you learn to say "No" decisively.
Decisions above my pay grade have us replacing queries via DNS [1] where we have to finish our work in max 1 second, with a nice, HTTPS REST API that returns JSON that is guaranteed to return within 1.7 seconds.
No one above my pay grade seems to see a problem with this. But hey! REST! JSON! HTTPS! Pass the Kool-Aid!
[1] NAPTR records---given a phone number, return name information; RFC-3401 to RFC-3404
I saw this at my last company. Even worse breaking it into microservices allowed teams of 2 or 3 to start building entire realms where only they could commit and used it for internal politics.
I witnessed someone that wanted to leverage their service into a promotion so they started pushing for an architecture where everything flowed through their service.
It was the slowest part of our stack and capped at 10tps.
People really drink the koolaid that is written on these sites and it is extremely detrimental to their companies. PostgreSQL with a nice boring Java/.NET layer would blow this stuff out of the water performance wise (for their actual real life usecase), would be far easier to manage, deploy, find people for etc. I mean; using these stacks is good for my wallet as advisor, but I have no clue why people do it when they are not even close to 1/100000th of Facebook.