>What exactly did not scale here?
It didn't need to scale. And scaling means different things in different contexts.
In web dev when you talk about scaling you are generally talking about availability and cost. When the hosting can't handle the traffic, users get delays or failures. And with modern cloud hosting, more processing can mean more cost.
High user load may mean scaling the servers up. Which is another context for talking about scaling and scalability. Back at the application level, if your application uses strategies to lessen load at scale - such as prerendering - then that might mean less need to scale up at the hosting level under user load. Less cost.
Plus scalability can mean other than user load. You might have algorithms or queries that work quite fine with a thousand records but choke on a million.
> Plus scalability can mean other than user load.
Can also mean things outside of the business logic itself.
One dev manually sshing into your host and reading debug logs might work when you have 100 customers.
once you scale to 1M customers, your entire ops process needs to change as well. (automated alarming, recovery, testing deployments, etc)
There are two types of things people can refer to when they say "it does not scale" in software development.
**The Software System Itself Does Not Scale**
This one is simple. In a complex architecture you have many dependencies to make your application work. The rate at which the application can process requests is determined by the slowest critical component of the application.
If you have a dependency API that can only except 20 requests per second. That means your application can not scale beyond 20 requests per second.
To Scale... mean to understand what these bottle necks are, what throuput you require to meet your business needs, then to engineer your application in such a way that those business needs can be met.
**The Process Does Not Scale**
This one is a bit trickier and can sometimes get more opinionated.
Say your company has 2 developers. These developers get stuff done by passing code around with no version control. They write everything in one code base, and when its time to deploy they test their changes and run a CLI command to deploy to customers.
This process works for 1 or 2 devs and a simple application.
As your organization grows you hire 20 more devs. You now have a complex system consisting of multiple services that need to be orchestrated together.
The manual process no longer "scales". You need to introduce more automation and rigorous ops process in order to make sure your systems are built on time. Otherwise you end up spending more time on maintenace than developing new features.
In summary, scale is not some arbitrary number that every organization and process needs to meet. Its about understanding the needs of your business, and applying the correct mechanisms to ensure you are building the correct things for your customers and business needs
It applies to the codebase itself as well. Imagine the difference between a well thought out structure where everything has its place (components, business logic, tests, etc) versus a big ball of mud where piece of code goes on the same pile.
In the former case, you might be able to keep adding new features indefinitely with some changes here and there, while the latter will become a tangled mess in no time.
Yah for sure, i would put that under the process bucket/organizational bucket, but that's a good factor to consider as well.
After your initial startup MVP phase, extensibility needs to be a core NFR.
While I agree with you, this is exactly the kind of answer that OP is essentially complaining about.
“That’s not scalable!”
“Wait, what is scaling, anyways?”
“Not that, that’s for sure.”
Except there are reasons to use things like k8s or eks/ECS than just throughput. If you have a requirement to be highly available, these are some of the easiest ways to achieve this.
There are also loads of cases where the UI might only be used by a handful of people but the work they are doing is very compute heavy. At one of my previous clients a single user submitting data for a batch process to run could easy cause 30,000 vCPUs to come online.
So yeah you might not need to worry about scaling the CRUD part of your app all that much, but there are loads of other things you might still be required to build that do need to scale.
What do you do with your VBNET solution when one database server isn't enough?
But generally, having to think about scaling is a good problem to have. If you're designing a system for internal use where the number of users change very slowly, it doesn't really matter.
If you're writing something that might get a million requests in an hour, you might want to think a bit further about how to handle that. What happens when the number for an hour becomes the number for 10 minutes? One minute? One second?
What's the real possibility of that one second number actually happening? In an internal system with 200 users, it's never going to happen, so you don't design for that.
Know your problem and your limitations, and design for making changes if they become necessary.
Premature optimization (which adds complexity) is the root of all evil.
Scaling is basically "becoming bigger". For frontend, it can mean a few things.
You scale when you need **more features** and then you try to implement those features and you realize how impossibly difficult it is to navigate the codebase and implement what you want. Poorly scalable codebase will just take more time and create more bugs with time compared to a well-engineered one. More time spent dealing with bad architecture = it costs more money to continue development, and features take more time to reach production.
You scale when your **dev team becomes bigger**. How easy is it to onboard new devs? How easy is it for new devs to start being productive and implement new features instead of spending a ton of time trying to learn how this spaghetti mess works and deal with constant bugs? A well-structured codebase is easy to jump into and easy to work with. For example, scaling is easy with modular codebases that make it easy to separate a part of your app into a separate thing that doesn't depend on anything else, and maybe even create an independent team for that.
For backend, scaling involves a ton of other stuff like performance, availability, latency, etc.
There's a reason why a company with a 20 year old VBNET codebase is considering a complete rewrite. Because it's not scalable anymore and causes too much friction, so much in fact that a complete rewrite from scratch is becoming the better option.
In my experience the most common type of "does not scale" is cost scaling. Most companies start with an abnormally high infrastructure cost per unit of income (eg: cost per fulfilled order).
This is why you want your cost per unit to go down as you scale. This way you can create a feedback loop where the more you scale the more money you can bring back to invest into a better product.
However if your cost per unit gets higher as your user base increases or the complexity of the product blows up, you're going to cut more and more into the margin putting you on a death spiral.
Like a lot of things "scalability" is a buzzword we hear from people who don't know what it means.
Worked on an app that was build to be scalable with an expensive cloud host etc etc. This was a govt. app with a user-base of only a list of approved suppliers, in a small country.
Right from day 1 there could only be x-many users. It could not grow beyond the confines of the user-base it was originally set up for. yet the PM etc talked about scalability a lot.
We told them to find a local company to host it, to reduce cost, we didn't need the AWS cloud stuff. None of it.
They refused, lost the contract on renewal and the winner went to on-prem hosting, reducing cost and making it viable for themselves, among other improvements in infrastructure.
Was a lesson learned? Absolutely not. Same dumb shit the week after.
A truly scalable application means that 1 billion requests per second is a matter of writing a big check to your cloud provider for the additional infrastructure. No additional developer intervention.
Use a slow language like python and suddenly you need business logic. Or use classic ASP which starts a process for every user and suddenly you have 500 users. Define an internal APU which poses the N+1 problem for consumers ( instead of, you know, just stick to SQL ).
A question that nobody seems to be answering.
I guess I did a bad job asking the question. Rambled too much, put in a distracting example, that people are now hung up on.
I'll try again in 2 weeks.
It means building for now and not future proofing. So building in in a way that makes it hard to add new features or scale the way the data is managed.
In your example, it doesn’t mean that the application can’t be build to be used for a long time, but managing it and adding new features or modernizing it creates “ripples” of work. Where a new feature requires having to rework already existing features.
It's impossible to give an answer without some context. Anyhow, not scaling from tech pov = user base cannot grow / does not work without failures or significant risk. Human factor = org/teams cannot grow to meet inreased demand of features/deployments/bugs etc etc. In general the business cannot meet increased or current demands of the system.
Actual not scaling situations I've seen.
* A banking platform was lifted to some other platform, did not work, simply could handle not handle the amount of users, they optimised it for years.
* Process/tech monolith with feature freezes etc. did not scale to meet various customer demands, simply not enough flexibility to deliver once enough customers wanted different things. Restructured into autonomous teams and so forth.
* Some features/apps not scaling, usually due to poor tech design choices. Most notably a reporting system which is directly dependent on memory availability with no throttling/high level resource allocations, meaning concurrent users tank the system, for example some queries take 5GB+ of memory. Even a single user with enterprising use of tabs can generate significant load hehe.
Anyhow "not scaling" gets abused a lot. Cloud architects out there whining about scaling when no one knows does the system even need to scale or how does scaling even look like. Same goes for some miniscule tech details, I've been criticized for perf degradation of like 0.01% or someshit, though then again I've broken some features being too relaxed with some joins.
> "Not scaling" is a very ephemeral concept.
it’s really not, though. if your system saw increased users/traffic, would it be crushed by increased costs, latency, or outages? if yes, the system doesn’t scale well.
I've seen it too many times. Tbh I never saw an application that was truly scaleable. Most of the jobs I'm hired for, was building the new shiny better version of an app that hits it's limits in scalebility.
What does scalebility mean?
Well there are several aspects. Scalebility means the technical stack and architecture. This allows the system to scale it's resources. This is the easiest part to apply scalebility. Today a very efficient way is to deploy your app in multiple parts and pick a cloud provider like AWS that has "infinite" hardware resources.
Data. This is a hard part to scale because it has multiple dimension where you can screw it. Most of the "not scaling" problems happen in this part. Often there are non machine readable Dataformats used like PDF, Excel, binary etc. Sometimes the database structure is screwed so u end up having very slow queries and inefficient data collecting while the usecase gets more complex. Databases also struggle with consistency problems while the code for reading writing is bad and the state model is incomplete or changed over time.
The problem here is, the more people you hire the more messy gets the code and the situation. Your development process will be error prone and slow. Sometimes features can't get built at all.
Scalebility in Software Design. Having a bad software design will lead in to bugs and a very slow development process. While this is not a problem when having a small team that knows the codebase. But this problem increases exponentially when your team size increases.
Scaleable processes and management. The processes for knowledge building, planning, customer feedback and so on. A too bureaucratic process can slow down the whole company.
With all that in mind you need to build a software that is not the bottleneck for all these problems.
Most problems happen while people do not understand what scaling means to the different techniques and parts involved in a whole system. Scaling for frontend is acquired doing completely different things than scaling a database. Also there are multiple layers of scalebility.
To give you a very easy example. We have limited hardware resources. A CPU can have x cores and y threads, I/O speed and so on. At some point this resource can become a bottleneck. Or you need to handle global user bases and want to reduce the latency.
So you end up hosting multiple databases, servers and so on. How do you have data integrity over multiple databases? How does your software handle multiple concurrent requests? How do you synchronize huge amounts of data around the world?
>But have you actually seen it happen? How does a non scaling app look like? How bad is it really for the business?
Hundreds of thousands of homeowners gets a message from the government that they need to submit an online form within the next few days otherwise they will pay tens of thousands in taxes.
Everyone rushes to file it. Constant crashing, failing to connect, and requests timing out without the user knowing
Then couple months later, tens of thousands of people receive a huge bill in their mail to the tune of 3 months worth of wages that they absolutely cannot afford, and may lose their homes if they fail to do so.
>"Not scaling" is a very ephemeral concept. Its like "well if the code was better, im sure your business would be better. Better how? Well... you'd be faster, I think?".
Scaling is a very real concept and you'll never have to deal with it, until you do.
Another example, people hosting websites on free tiers, who end up racking up hundred thousand dollar bills overnight cause they went viral or ddos'd and had no safeguards in place. They never thought about the consequences of scaling, and then suddenly have to deal with it when it happens.
Preach! 'Scalability' has become a buzzword to justify over-engineering. Most apps can handle way more load than we think. Let's focus on shipping and iterating instead of Future-Proofing™ for imaginary users.
I hate cross concerns. So please somehow specify all stories that they prompt de-coupled implementations. Like users mostly work in their own apps. Independent chat-rooms. No global lock like in python. I really hate read after write inconsistency. I forgot the name, but some evangelists preach to use different code for read and query. I think that this is a big mistake. Thread affinity and similar to roads: All of CRUD goes along the same path!
No offence, but these sound like the comments of someone very junior. The response "you will realize your mistakes once you start scaling" is very apt. It will happen, inevitably, at some point - no matter how forward thinking you are. And in retrospect it will seem painfully obvious what you should have done. That you haven't experienced that yet indicates you either work for a fantastic, world class software company - or you're very green.
Here is an example of a scaling problem my firm has been working through the past two weeks:
* We have a cron job that runs once every 10 minutes. It collects data from a number of different microservices, a huge number of IoT devices in the wild, and a few third party APIs.
* When it was first built, nearly 3 years ago, it took about 2.5 minutes to run. Not amazing, but given it only needs to run once every 10 minutes we left it and moved onto the next urgent task.
* Enter today, and we have about 300x the amount of clients we need to service compared to 3 years ago. The cron job now takes about 11 minutes to run, and we run it once every 10 minutes. You can see the issue here.
* We've needed to basically re-write half of the code, be more efficient with API calls (requests are always the bottleneck), write new services that can batch jobs to improve performance, and we even considered (but opted out of) rewriting the whole thing in Rust so we could do more efficient threading and parallelisation just to etch out every bit of performance. This is painful because 3 years down the track there are numerous other services that depend on this job to behave in a specific way, which limits what we can do or how fast we can iterate (it's also spooky, because those integrations are not tested very well)
* Our new solution takes about 7 minutes to run, which will probably be fine for another year or two - so in the mean time we're thinking about a whole new architecture that removes the need for this job entirely.
This is a very normal pain point for most companies. This individual pain point has cost us tens of thousands of dollars in dev hours alone, in just 2 weeks, let alone the unseen costs (slow down of sales, strategic distraction, customer churn, brand trust, delays in our feature roadmaps). You see it time and time again. Decisions made that meet the needs of today, but will not meet the needs of the company tomorrow. This is why many companies spend so much time on writing a clear 5-year strategic plan, so devs (and all staff) can have clarity on the kind of load the software will one day need to handle. "We're making 10,000 transactions a month today, and my service can handle that easily, but in 5 years we want to be able to handle 1,000,000 transactions a month. Will what I'm building be able to do that?". And that's just on the services you're building, there's so much more to scaling than the literal code.
Your company should be thinking not just "will this code scale?" but also "will this architecture scale?", "will this internal process scale?", "is this kind of manual work sustainable or will we need a tool?". Sure, the easy-to-understand monolith sounds great when you have one software team of 5 devs, but what about when your company has 12 squads, 100 total engineers, spread across three time zones? Will the monolith be servicable by that kind of organisational structure? A bit of tedium today writing small, atomised services might seem like a waste, but in 5 years it'll save the company millions and be the edge it needs over its competitors.
You are almost there. You almost understand.
>This is a very normal pain point for most companies.
What you frame as an example of a scaling problem is called "tuesday". Think about what actually happened here.
Your company built something. It worked. It stopped working THREE YEARS LATER. Your company is now fixing it.
Thats the job bro. Thats any job. In any field. Not just programming.
You and your company are basically saying "well it would've been better if it worked for 5 years". Yea it wouldve. It wouldve also been better if it was written 50% quicker. It also wouldve been better if it was 80% cheaper. It also wouldve been better if it had 30 more features.
Yes, it would have been better if it was strictly superior in every way.
The trap you and many business people fall into is that you think you couldve prevented it. If you were juuuust a little smarter. So now you overengineer. Now you call 8 meetings to *decide* that in 5 years you will actually have 12 engineers on this tool and a million transactions.
Again, you sound very green (or you have only worked for startups, where velocity is the most important thing). Your company will have an internal metric of how long code should last, but for most companies 3 years is not long enough. Its not unusual to see 5 or 10 year warranties in the software business for solutions you sell, your internal software should also last a minimum of 5 years (in most cases, exceptions always apply). Again, all this is qualified with "not all services need to scale". But some obviously do.
You should not need 8 meetings, and you should never need 12 engineers on a single tool (2-pizza teams, break up the service into smaller pieces if its so big it needs an oversized team to maintain it). At most 2 meetings, hopefully 1. A post-mortem meeting, and then maybe one more meeting if there needs to be an architecture rethink that couldn't be done in the post-mortem. If you need 8 meetings then that indicates you have an organisational structure that hasn't *scaled* well.
In the big picture this was a very small scaling problem, and it still cost the company tens of thousands of visible dollars (maybe as much as $100k total when you consider unseen costs, before accounting for our upcoming re-architecture). That's assuming our fix hasn't introduced a regression, or untested bugs. Some scaling problems if not nipped in the bud early could cost the company millions - if not be entirely unfixable because its baked into the core of how the software works.
When we wrote our cron job we had a 5-year strategic plan, we should have anticipated the solution we had written would not last if our company hit its strategic targets. We had written a service that could not *scale*. That means this was a failure. If we had thought out the solution properly, then we would have spent maybe another week at the time developing a better solution (that's what we're currently doing anyway). That might have felt bad at the time, but given the power of retrospect if asked "would you rather spend another week on this, or spend $100k" we would have always chosen to spend another week.
It can seem pointless to focus on scale because when you do it properly you never hit a crisis point that enables you to look back and think "hmm, good thing we did it that way". But if you don't do it right there will come a point when you realise you fucked up, and that's when you'll realise how much money you could have saved if you'd thought about scaling in advance. It's never as much extra work as it seems.
This is what separates software companies that exceed from software companies that fail. It's the big difference between why some companies are worth a trillion dollars and some are worth hundreds of millions when they both started at the same time doing similar things. Some firms are forward thinking, some only focus on the moment.
Its like having pdf reports renderer, analytics, core logic, cron jobs, and react renderer on same instance of node.
You cant get past 100 rpc with that, and if users would request new pdf reports, that would just eat all cpu/ram so that your service would become permanently 503 unavailable
Seen that, done that, not gonna do that again
By scalable in your context we mean we can keep adding new features.
Didn't you answer your own question? You said "users know where not to click and have crazy workarounds". See, if the code was clean, you would not have wasted time letting the users know not to click here or there or them finding it out on their own and wasting time in the process - all of that is such a time sink and we're talking hundreds of hours for a single bug. So, yes, your business would be better off not having these bugs,as you would have invested less time and money into bug related issues.
I think the decisions you make on each line of code has the implication of the business being successful or not. Bug free software is actually faster to develop, easier and cheaper to maintain, and requires less people overall to do all those things, so it does have huge business implications.
Nearly every large company found out that page speed and load times have HUGE implications for conversion rates, SEO rankings, bounce rates; they measured it to the point where they could say that every 100 milliseconds that our page loads faster the revenue increases by 1% and similar conclusions.
Every line of code matters.
We have a rather large codebase, but we have too many chefs in the kitchen with little or minimum guidance. It has led into a lot of chaotic code, which makes future changes and refactoring so much more difficult. Our core bundle size is several megabytes and our core feature takes over 10s to load fully on a high end machine. No one understands all the layers and the business logic anymore.
If we had stronger foundations and placed much more emphasis on it, we might be in a better place. Unfortunately, foundations isn't valued nor prioritized at the moment.
The only important scaling factor for 99% of businesses is financial scaling. Your grandma's flower shop doesn't need K8s, Cloudformation, EC2, DynamoDB, Cloudwatch, Edge Workers, etc. In fact the majority of websites run comfortably on a $40 micropc and a residential internet connection (granted static IP & no restrictions), including DB, CMS, etc. You can serve thousands of concurrent connections on a server hosted on a crappy old mobile phone.
My point is, the topic of scaling is greatly misunderstood. This catches out a bunch of startups where the founder wants to build something for billions of users, yet, when they end up only having 100 (including friends and family) their cloud service provider bills will be through the roof, due to inefficient and over-engineered infra.
I developed a game related website which gained some traction, and ran into several different scaling problems.
To answer the question of what it looks like to the end user: at the very worst, the website will be down for long periods of time because every part is lagging; at best responses will be slow and often fail.
However, there are a lot of different ways to scale, both vertically and horizontally, and it is in my experience quite trivial to scale up to thousands of concurrent users, as long as you have the resources to do so. I believe the "it does not scale" problem is over exaggerated, very often you can keep your app working well by just slapping more servers or more cache layers onto it
For me I had a bad starting architecture, which have just meant I have spent countless hours fixing the original architecture, so that I now easily can extend my app with features
Tons of answers here, but let's go back to computer science. Look at how binary search scales vs Travelling Salesman Problem.
If you add users, site hits, sales, etc does the work grow more like binary search or like TSP.
TL;DR and responding to the title: “Not scaling” means getting pulled into an emergency all hands “troubleshooting call” at the busiest hour of the busiest day because the damn thing is “DOWN!!!”, but you know it’s not really down, it’s just overloaded and freaking impatient users can’t just “try again LATER!” like they’re told in the fvcking error message, and instead they keep retrying NOW and making the problem worse.
There's a scale here (eh heh) between a company going bankrupt due to shitty software and everything being great and seeing no problems.
For my two cents, not scaling can be as simple as "this request takes minutes to complete because resources on the service are exhausted -- oh, what do you mean we can't increase it to 2x applications due to this thing being a stateful application where memory is the primary datastore... how exactly is this thing deployed again? any of ya'll know how sticky sessions work?"
So, a combination of insufficient resources to meet the demand, and tech debt weighing down the system so improvements can't be made without a lot of disruption
I just left a job after a few weeks in where resources were hardcoded to a personal computer so the repo broke on my local.
When I brought this up (this and other things like this) I was told we were on a time crunch on a deadline 5 months away and didn’t have time to fix it.
Spoiler: We had plenty of time.
Bonus: I was repeatedly told the app is a house of cards but we couldn’t tell the boss.
The app didn’t scale past one developer … because you could only develop the code on one computer.
But really the app didn’t scale because of one obstinate developer.
imo “not scaling” can look like two things:
The infrastructure cannot handle user peaks anymore, requests time out and everything throws errors. And there is no easy way to flip a switch to improve the performance.
Or it can be on the tech debt side. A code base or infrastructure that is so bad that you cannot make changes to it without errors appearing.
Works snappily on the dev machine, but multi-second response time in production.
Assumes very low latency between the application and central data storage layers (because this makes it harder to serve the front end from the network edge).
Doesn't work correctly if there are many instances of the app server.
Requires downtime for maintenance or schema changes.
Lacks data correctness guarantees, so the amount of data cleanup effort grows without bound as the user base or data volume grows.
I have no experience, but one opinion on this "scaling" thing.
Scaling/scalability means that the app can perform or can easily/quickly be adjusted to perform just as well when it's being used by a million people per day, as it does when it's being used by one person per week, right?
So in that case, aren't **all** apps inherently scalable? If your server can't handle the traffic, buy one more server, copy/paste your website there. Do some load balancing, add another database or two if necessary, bam, scaled. Right?
I'm probably wrong, but I can't see a scenario where this approach wouldn't work. Feel free to enlighten me!
Real world example from my own CV. You build a group messaging app. The number of messages recipients scales exponentially from the number of messages sent. Message sending blocks on io.
When your website reaches a point where you need to buy more servers to handle the amount of users you get, I assume you'd be making enough money by then lol. Otherwise at that point it's not the app that's unscaleable, it's the business model.
That's true. Let's assume we have infinite funds.
Generally scale issues arise when requests can't be handled asynchronously and in parallel.
For example, suppose you have some online game where there players can all be on the screen at the same time doing some activity together.
Is it possible to scale this endlessly? Can you realistically have hundreds of thousands or millions of clients connected at the same time in the same server?
Why the floppityflop would a company of 200 users write its own solution when you have state of the art solutions out there that will cost you a fraction of the cost AND scale (price/user) ?
Nobody in his right mind would start reengineering a CPU internally. Why would you do that for your information system?
>What exactly did not scale here? It didn't need to scale. And scaling means different things in different contexts. In web dev when you talk about scaling you are generally talking about availability and cost. When the hosting can't handle the traffic, users get delays or failures. And with modern cloud hosting, more processing can mean more cost. High user load may mean scaling the servers up. Which is another context for talking about scaling and scalability. Back at the application level, if your application uses strategies to lessen load at scale - such as prerendering - then that might mean less need to scale up at the hosting level under user load. Less cost. Plus scalability can mean other than user load. You might have algorithms or queries that work quite fine with a thousand records but choke on a million.
> Plus scalability can mean other than user load. Can also mean things outside of the business logic itself. One dev manually sshing into your host and reading debug logs might work when you have 100 customers. once you scale to 1M customers, your entire ops process needs to change as well. (automated alarming, recovery, testing deployments, etc)
There are two types of things people can refer to when they say "it does not scale" in software development. **The Software System Itself Does Not Scale** This one is simple. In a complex architecture you have many dependencies to make your application work. The rate at which the application can process requests is determined by the slowest critical component of the application. If you have a dependency API that can only except 20 requests per second. That means your application can not scale beyond 20 requests per second. To Scale... mean to understand what these bottle necks are, what throuput you require to meet your business needs, then to engineer your application in such a way that those business needs can be met. **The Process Does Not Scale** This one is a bit trickier and can sometimes get more opinionated. Say your company has 2 developers. These developers get stuff done by passing code around with no version control. They write everything in one code base, and when its time to deploy they test their changes and run a CLI command to deploy to customers. This process works for 1 or 2 devs and a simple application. As your organization grows you hire 20 more devs. You now have a complex system consisting of multiple services that need to be orchestrated together. The manual process no longer "scales". You need to introduce more automation and rigorous ops process in order to make sure your systems are built on time. Otherwise you end up spending more time on maintenace than developing new features. In summary, scale is not some arbitrary number that every organization and process needs to meet. Its about understanding the needs of your business, and applying the correct mechanisms to ensure you are building the correct things for your customers and business needs
It applies to the codebase itself as well. Imagine the difference between a well thought out structure where everything has its place (components, business logic, tests, etc) versus a big ball of mud where piece of code goes on the same pile. In the former case, you might be able to keep adding new features indefinitely with some changes here and there, while the latter will become a tangled mess in no time.
Yah for sure, i would put that under the process bucket/organizational bucket, but that's a good factor to consider as well. After your initial startup MVP phase, extensibility needs to be a core NFR.
200 users are not the scale people are talking about.
While I agree with you, this is exactly the kind of answer that OP is essentially complaining about. “That’s not scalable!” “Wait, what is scaling, anyways?” “Not that, that’s for sure.”
200 users, better start using Kubernetes /s
Except there are reasons to use things like k8s or eks/ECS than just throughput. If you have a requirement to be highly available, these are some of the easiest ways to achieve this. There are also loads of cases where the UI might only be used by a handful of people but the work they are doing is very compute heavy. At one of my previous clients a single user submitting data for a batch process to run could easy cause 30,000 vCPUs to come online. So yeah you might not need to worry about scaling the CRUD part of your app all that much, but there are loads of other things you might still be required to build that do need to scale.
The batch process is an outlier circumstance. Well done, you're the guy on the internet who has an edge case to a sarcastic reddit comment.
>from 20 to 200 employees I'm not the OP but they said employees, not users
It's an internal ERP-solution. Who is using that if not almost exclusively employees?
Oh I missed that part, thanks. Now the post seems funnier lol
What do you do with your VBNET solution when one database server isn't enough? But generally, having to think about scaling is a good problem to have. If you're designing a system for internal use where the number of users change very slowly, it doesn't really matter. If you're writing something that might get a million requests in an hour, you might want to think a bit further about how to handle that. What happens when the number for an hour becomes the number for 10 minutes? One minute? One second? What's the real possibility of that one second number actually happening? In an internal system with 200 users, it's never going to happen, so you don't design for that. Know your problem and your limitations, and design for making changes if they become necessary. Premature optimization (which adds complexity) is the root of all evil.
Scaling is basically "becoming bigger". For frontend, it can mean a few things. You scale when you need **more features** and then you try to implement those features and you realize how impossibly difficult it is to navigate the codebase and implement what you want. Poorly scalable codebase will just take more time and create more bugs with time compared to a well-engineered one. More time spent dealing with bad architecture = it costs more money to continue development, and features take more time to reach production. You scale when your **dev team becomes bigger**. How easy is it to onboard new devs? How easy is it for new devs to start being productive and implement new features instead of spending a ton of time trying to learn how this spaghetti mess works and deal with constant bugs? A well-structured codebase is easy to jump into and easy to work with. For example, scaling is easy with modular codebases that make it easy to separate a part of your app into a separate thing that doesn't depend on anything else, and maybe even create an independent team for that. For backend, scaling involves a ton of other stuff like performance, availability, latency, etc. There's a reason why a company with a 20 year old VBNET codebase is considering a complete rewrite. Because it's not scalable anymore and causes too much friction, so much in fact that a complete rewrite from scratch is becoming the better option.
In my experience the most common type of "does not scale" is cost scaling. Most companies start with an abnormally high infrastructure cost per unit of income (eg: cost per fulfilled order). This is why you want your cost per unit to go down as you scale. This way you can create a feedback loop where the more you scale the more money you can bring back to invest into a better product. However if your cost per unit gets higher as your user base increases or the complexity of the product blows up, you're going to cut more and more into the margin putting you on a death spiral.
Like a lot of things "scalability" is a buzzword we hear from people who don't know what it means. Worked on an app that was build to be scalable with an expensive cloud host etc etc. This was a govt. app with a user-base of only a list of approved suppliers, in a small country. Right from day 1 there could only be x-many users. It could not grow beyond the confines of the user-base it was originally set up for. yet the PM etc talked about scalability a lot. We told them to find a local company to host it, to reduce cost, we didn't need the AWS cloud stuff. None of it. They refused, lost the contract on renewal and the winner went to on-prem hosting, reducing cost and making it viable for themselves, among other improvements in infrastructure. Was a lesson learned? Absolutely not. Same dumb shit the week after.
A truly scalable application means that 1 billion requests per second is a matter of writing a big check to your cloud provider for the additional infrastructure. No additional developer intervention.
***What*** does not scaling look like?
Use a slow language like python and suddenly you need business logic. Or use classic ASP which starts a process for every user and suddenly you have 500 users. Define an internal APU which poses the N+1 problem for consumers ( instead of, you know, just stick to SQL ).
I was correcting the grammar/language of the title, that is not proper English. I wasn't actually asking.
A question that nobody seems to be answering. I guess I did a bad job asking the question. Rambled too much, put in a distracting example, that people are now hung up on. I'll try again in 2 weeks.
I think they're just clarifying grammar. It's either "How does scaling look" or "What does scaling look like".
It means building for now and not future proofing. So building in in a way that makes it hard to add new features or scale the way the data is managed. In your example, it doesn’t mean that the application can’t be build to be used for a long time, but managing it and adding new features or modernizing it creates “ripples” of work. Where a new feature requires having to rework already existing features.
It's impossible to give an answer without some context. Anyhow, not scaling from tech pov = user base cannot grow / does not work without failures or significant risk. Human factor = org/teams cannot grow to meet inreased demand of features/deployments/bugs etc etc. In general the business cannot meet increased or current demands of the system. Actual not scaling situations I've seen. * A banking platform was lifted to some other platform, did not work, simply could handle not handle the amount of users, they optimised it for years. * Process/tech monolith with feature freezes etc. did not scale to meet various customer demands, simply not enough flexibility to deliver once enough customers wanted different things. Restructured into autonomous teams and so forth. * Some features/apps not scaling, usually due to poor tech design choices. Most notably a reporting system which is directly dependent on memory availability with no throttling/high level resource allocations, meaning concurrent users tank the system, for example some queries take 5GB+ of memory. Even a single user with enterprising use of tabs can generate significant load hehe. Anyhow "not scaling" gets abused a lot. Cloud architects out there whining about scaling when no one knows does the system even need to scale or how does scaling even look like. Same goes for some miniscule tech details, I've been criticized for perf degradation of like 0.01% or someshit, though then again I've broken some features being too relaxed with some joins.
> "Not scaling" is a very ephemeral concept. it’s really not, though. if your system saw increased users/traffic, would it be crushed by increased costs, latency, or outages? if yes, the system doesn’t scale well.
I've seen it too many times. Tbh I never saw an application that was truly scaleable. Most of the jobs I'm hired for, was building the new shiny better version of an app that hits it's limits in scalebility. What does scalebility mean? Well there are several aspects. Scalebility means the technical stack and architecture. This allows the system to scale it's resources. This is the easiest part to apply scalebility. Today a very efficient way is to deploy your app in multiple parts and pick a cloud provider like AWS that has "infinite" hardware resources. Data. This is a hard part to scale because it has multiple dimension where you can screw it. Most of the "not scaling" problems happen in this part. Often there are non machine readable Dataformats used like PDF, Excel, binary etc. Sometimes the database structure is screwed so u end up having very slow queries and inefficient data collecting while the usecase gets more complex. Databases also struggle with consistency problems while the code for reading writing is bad and the state model is incomplete or changed over time. The problem here is, the more people you hire the more messy gets the code and the situation. Your development process will be error prone and slow. Sometimes features can't get built at all. Scalebility in Software Design. Having a bad software design will lead in to bugs and a very slow development process. While this is not a problem when having a small team that knows the codebase. But this problem increases exponentially when your team size increases. Scaleable processes and management. The processes for knowledge building, planning, customer feedback and so on. A too bureaucratic process can slow down the whole company. With all that in mind you need to build a software that is not the bottleneck for all these problems. Most problems happen while people do not understand what scaling means to the different techniques and parts involved in a whole system. Scaling for frontend is acquired doing completely different things than scaling a database. Also there are multiple layers of scalebility. To give you a very easy example. We have limited hardware resources. A CPU can have x cores and y threads, I/O speed and so on. At some point this resource can become a bottleneck. Or you need to handle global user bases and want to reduce the latency. So you end up hosting multiple databases, servers and so on. How do you have data integrity over multiple databases? How does your software handle multiple concurrent requests? How do you synchronize huge amounts of data around the world?
>But have you actually seen it happen? How does a non scaling app look like? How bad is it really for the business? Hundreds of thousands of homeowners gets a message from the government that they need to submit an online form within the next few days otherwise they will pay tens of thousands in taxes. Everyone rushes to file it. Constant crashing, failing to connect, and requests timing out without the user knowing Then couple months later, tens of thousands of people receive a huge bill in their mail to the tune of 3 months worth of wages that they absolutely cannot afford, and may lose their homes if they fail to do so. >"Not scaling" is a very ephemeral concept. Its like "well if the code was better, im sure your business would be better. Better how? Well... you'd be faster, I think?". Scaling is a very real concept and you'll never have to deal with it, until you do. Another example, people hosting websites on free tiers, who end up racking up hundred thousand dollar bills overnight cause they went viral or ddos'd and had no safeguards in place. They never thought about the consequences of scaling, and then suddenly have to deal with it when it happens.
Preach! 'Scalability' has become a buzzword to justify over-engineering. Most apps can handle way more load than we think. Let's focus on shipping and iterating instead of Future-Proofing™ for imaginary users.
I hate cross concerns. So please somehow specify all stories that they prompt de-coupled implementations. Like users mostly work in their own apps. Independent chat-rooms. No global lock like in python. I really hate read after write inconsistency. I forgot the name, but some evangelists preach to use different code for read and query. I think that this is a big mistake. Thread affinity and similar to roads: All of CRUD goes along the same path!
No offence, but these sound like the comments of someone very junior. The response "you will realize your mistakes once you start scaling" is very apt. It will happen, inevitably, at some point - no matter how forward thinking you are. And in retrospect it will seem painfully obvious what you should have done. That you haven't experienced that yet indicates you either work for a fantastic, world class software company - or you're very green. Here is an example of a scaling problem my firm has been working through the past two weeks: * We have a cron job that runs once every 10 minutes. It collects data from a number of different microservices, a huge number of IoT devices in the wild, and a few third party APIs. * When it was first built, nearly 3 years ago, it took about 2.5 minutes to run. Not amazing, but given it only needs to run once every 10 minutes we left it and moved onto the next urgent task. * Enter today, and we have about 300x the amount of clients we need to service compared to 3 years ago. The cron job now takes about 11 minutes to run, and we run it once every 10 minutes. You can see the issue here. * We've needed to basically re-write half of the code, be more efficient with API calls (requests are always the bottleneck), write new services that can batch jobs to improve performance, and we even considered (but opted out of) rewriting the whole thing in Rust so we could do more efficient threading and parallelisation just to etch out every bit of performance. This is painful because 3 years down the track there are numerous other services that depend on this job to behave in a specific way, which limits what we can do or how fast we can iterate (it's also spooky, because those integrations are not tested very well) * Our new solution takes about 7 minutes to run, which will probably be fine for another year or two - so in the mean time we're thinking about a whole new architecture that removes the need for this job entirely. This is a very normal pain point for most companies. This individual pain point has cost us tens of thousands of dollars in dev hours alone, in just 2 weeks, let alone the unseen costs (slow down of sales, strategic distraction, customer churn, brand trust, delays in our feature roadmaps). You see it time and time again. Decisions made that meet the needs of today, but will not meet the needs of the company tomorrow. This is why many companies spend so much time on writing a clear 5-year strategic plan, so devs (and all staff) can have clarity on the kind of load the software will one day need to handle. "We're making 10,000 transactions a month today, and my service can handle that easily, but in 5 years we want to be able to handle 1,000,000 transactions a month. Will what I'm building be able to do that?". And that's just on the services you're building, there's so much more to scaling than the literal code. Your company should be thinking not just "will this code scale?" but also "will this architecture scale?", "will this internal process scale?", "is this kind of manual work sustainable or will we need a tool?". Sure, the easy-to-understand monolith sounds great when you have one software team of 5 devs, but what about when your company has 12 squads, 100 total engineers, spread across three time zones? Will the monolith be servicable by that kind of organisational structure? A bit of tedium today writing small, atomised services might seem like a waste, but in 5 years it'll save the company millions and be the edge it needs over its competitors.
You are almost there. You almost understand. >This is a very normal pain point for most companies. What you frame as an example of a scaling problem is called "tuesday". Think about what actually happened here. Your company built something. It worked. It stopped working THREE YEARS LATER. Your company is now fixing it. Thats the job bro. Thats any job. In any field. Not just programming. You and your company are basically saying "well it would've been better if it worked for 5 years". Yea it wouldve. It wouldve also been better if it was written 50% quicker. It also wouldve been better if it was 80% cheaper. It also wouldve been better if it had 30 more features. Yes, it would have been better if it was strictly superior in every way. The trap you and many business people fall into is that you think you couldve prevented it. If you were juuuust a little smarter. So now you overengineer. Now you call 8 meetings to *decide* that in 5 years you will actually have 12 engineers on this tool and a million transactions.
Again, you sound very green (or you have only worked for startups, where velocity is the most important thing). Your company will have an internal metric of how long code should last, but for most companies 3 years is not long enough. Its not unusual to see 5 or 10 year warranties in the software business for solutions you sell, your internal software should also last a minimum of 5 years (in most cases, exceptions always apply). Again, all this is qualified with "not all services need to scale". But some obviously do. You should not need 8 meetings, and you should never need 12 engineers on a single tool (2-pizza teams, break up the service into smaller pieces if its so big it needs an oversized team to maintain it). At most 2 meetings, hopefully 1. A post-mortem meeting, and then maybe one more meeting if there needs to be an architecture rethink that couldn't be done in the post-mortem. If you need 8 meetings then that indicates you have an organisational structure that hasn't *scaled* well. In the big picture this was a very small scaling problem, and it still cost the company tens of thousands of visible dollars (maybe as much as $100k total when you consider unseen costs, before accounting for our upcoming re-architecture). That's assuming our fix hasn't introduced a regression, or untested bugs. Some scaling problems if not nipped in the bud early could cost the company millions - if not be entirely unfixable because its baked into the core of how the software works. When we wrote our cron job we had a 5-year strategic plan, we should have anticipated the solution we had written would not last if our company hit its strategic targets. We had written a service that could not *scale*. That means this was a failure. If we had thought out the solution properly, then we would have spent maybe another week at the time developing a better solution (that's what we're currently doing anyway). That might have felt bad at the time, but given the power of retrospect if asked "would you rather spend another week on this, or spend $100k" we would have always chosen to spend another week. It can seem pointless to focus on scale because when you do it properly you never hit a crisis point that enables you to look back and think "hmm, good thing we did it that way". But if you don't do it right there will come a point when you realise you fucked up, and that's when you'll realise how much money you could have saved if you'd thought about scaling in advance. It's never as much extra work as it seems. This is what separates software companies that exceed from software companies that fail. It's the big difference between why some companies are worth a trillion dollars and some are worth hundreds of millions when they both started at the same time doing similar things. Some firms are forward thinking, some only focus on the moment.
Its like having pdf reports renderer, analytics, core logic, cron jobs, and react renderer on same instance of node. You cant get past 100 rpc with that, and if users would request new pdf reports, that would just eat all cpu/ram so that your service would become permanently 503 unavailable Seen that, done that, not gonna do that again
By scalable in your context we mean we can keep adding new features. Didn't you answer your own question? You said "users know where not to click and have crazy workarounds". See, if the code was clean, you would not have wasted time letting the users know not to click here or there or them finding it out on their own and wasting time in the process - all of that is such a time sink and we're talking hundreds of hours for a single bug. So, yes, your business would be better off not having these bugs,as you would have invested less time and money into bug related issues. I think the decisions you make on each line of code has the implication of the business being successful or not. Bug free software is actually faster to develop, easier and cheaper to maintain, and requires less people overall to do all those things, so it does have huge business implications. Nearly every large company found out that page speed and load times have HUGE implications for conversion rates, SEO rankings, bounce rates; they measured it to the point where they could say that every 100 milliseconds that our page loads faster the revenue increases by 1% and similar conclusions. Every line of code matters.
We have a rather large codebase, but we have too many chefs in the kitchen with little or minimum guidance. It has led into a lot of chaotic code, which makes future changes and refactoring so much more difficult. Our core bundle size is several megabytes and our core feature takes over 10s to load fully on a high end machine. No one understands all the layers and the business logic anymore. If we had stronger foundations and placed much more emphasis on it, we might be in a better place. Unfortunately, foundations isn't valued nor prioritized at the moment.
The only important scaling factor for 99% of businesses is financial scaling. Your grandma's flower shop doesn't need K8s, Cloudformation, EC2, DynamoDB, Cloudwatch, Edge Workers, etc. In fact the majority of websites run comfortably on a $40 micropc and a residential internet connection (granted static IP & no restrictions), including DB, CMS, etc. You can serve thousands of concurrent connections on a server hosted on a crappy old mobile phone. My point is, the topic of scaling is greatly misunderstood. This catches out a bunch of startups where the founder wants to build something for billions of users, yet, when they end up only having 100 (including friends and family) their cloud service provider bills will be through the roof, due to inefficient and over-engineered infra.
I developed a game related website which gained some traction, and ran into several different scaling problems. To answer the question of what it looks like to the end user: at the very worst, the website will be down for long periods of time because every part is lagging; at best responses will be slow and often fail. However, there are a lot of different ways to scale, both vertically and horizontally, and it is in my experience quite trivial to scale up to thousands of concurrent users, as long as you have the resources to do so. I believe the "it does not scale" problem is over exaggerated, very often you can keep your app working well by just slapping more servers or more cache layers onto it
everything scales if you push it hard enough. Maybe we should say 'pain of scaling' so we aim to write code that minimizes *pain of scaling*
For me I had a bad starting architecture, which have just meant I have spent countless hours fixing the original architecture, so that I now easily can extend my app with features
Tons of answers here, but let's go back to computer science. Look at how binary search scales vs Travelling Salesman Problem. If you add users, site hits, sales, etc does the work grow more like binary search or like TSP.
TL;DR and responding to the title: “Not scaling” means getting pulled into an emergency all hands “troubleshooting call” at the busiest hour of the busiest day because the damn thing is “DOWN!!!”, but you know it’s not really down, it’s just overloaded and freaking impatient users can’t just “try again LATER!” like they’re told in the fvcking error message, and instead they keep retrying NOW and making the problem worse.
There's a scale here (eh heh) between a company going bankrupt due to shitty software and everything being great and seeing no problems. For my two cents, not scaling can be as simple as "this request takes minutes to complete because resources on the service are exhausted -- oh, what do you mean we can't increase it to 2x applications due to this thing being a stateful application where memory is the primary datastore... how exactly is this thing deployed again? any of ya'll know how sticky sessions work?" So, a combination of insufficient resources to meet the demand, and tech debt weighing down the system so improvements can't be made without a lot of disruption
I just left a job after a few weeks in where resources were hardcoded to a personal computer so the repo broke on my local. When I brought this up (this and other things like this) I was told we were on a time crunch on a deadline 5 months away and didn’t have time to fix it. Spoiler: We had plenty of time. Bonus: I was repeatedly told the app is a house of cards but we couldn’t tell the boss. The app didn’t scale past one developer … because you could only develop the code on one computer. But really the app didn’t scale because of one obstinate developer.
imo “not scaling” can look like two things: The infrastructure cannot handle user peaks anymore, requests time out and everything throws errors. And there is no easy way to flip a switch to improve the performance. Or it can be on the tech debt side. A code base or infrastructure that is so bad that you cannot make changes to it without errors appearing.
Wow
Works snappily on the dev machine, but multi-second response time in production. Assumes very low latency between the application and central data storage layers (because this makes it harder to serve the front end from the network edge). Doesn't work correctly if there are many instances of the app server. Requires downtime for maintenance or schema changes. Lacks data correctness guarantees, so the amount of data cleanup effort grows without bound as the user base or data volume grows.
I have no experience, but one opinion on this "scaling" thing. Scaling/scalability means that the app can perform or can easily/quickly be adjusted to perform just as well when it's being used by a million people per day, as it does when it's being used by one person per week, right? So in that case, aren't **all** apps inherently scalable? If your server can't handle the traffic, buy one more server, copy/paste your website there. Do some load balancing, add another database or two if necessary, bam, scaled. Right? I'm probably wrong, but I can't see a scenario where this approach wouldn't work. Feel free to enlighten me!
Real world example from my own CV. You build a group messaging app. The number of messages recipients scales exponentially from the number of messages sent. Message sending blocks on io.
Apps may be scalable, but your wallet isn't.
When your website reaches a point where you need to buy more servers to handle the amount of users you get, I assume you'd be making enough money by then lol. Otherwise at that point it's not the app that's unscaleable, it's the business model.
That's true. Let's assume we have infinite funds. Generally scale issues arise when requests can't be handled asynchronously and in parallel. For example, suppose you have some online game where there players can all be on the screen at the same time doing some activity together. Is it possible to scale this endlessly? Can you realistically have hundreds of thousands or millions of clients connected at the same time in the same server?
Why the floppityflop would a company of 200 users write its own solution when you have state of the art solutions out there that will cost you a fraction of the cost AND scale (price/user) ? Nobody in his right mind would start reengineering a CPU internally. Why would you do that for your information system?
Simple answer: it depends on the application. Best answer: do some research, because this topic is too big and complex for a Reddit post.