How to optimize and scale Meteor projects? Guidelines based on our experience.
In case your Meteor app is nearing its limits of traffic it can support, your user experience is hindered by performance, or in case your server costs eat up into your budget, most likely you have some options to optimize your project. You can also scale. In this article, we describe low hanging fruits related to both those approaches, based on our long experience of planning such optimizations.
Check the performance of your app with performance with Kadira APM
Kadira APM is one of the tools that you can use to optimize your live queries and loading time, which ultimately boosts the performance of your Meteor app. Live queries are responsible for changing data automatically in real-time, and they have a great impact on your app’s CPU usage. The count of queries does not affect performance, but different factors such as the number of documents fetched by Live Queries, the number of live changes, and oplog notifications received by Meteor, impact CPU usage.
If you go to Meteor APM Live Queries tab, you can sort the publications by “fetched documents”. It would be great if you could reduce the number of documents fetched. It can be done by changing your code and setting a limit when you fetch data.
Each live query comes with an internal observer responsible for watching changes in the DB and notifying the Live Query. Similar queries can stay under the same observers, as long as the query and different options passed to Collection.find() (e.g., fields, sort) are the same. If your observer reuse ratio is high, you have managed to optimize your live queries nicely. What to do with busy live queries? Firstly identify subscription triggering many events, and if possible, reduce changes happening in those queries, for example, using a field filter when applicable.
Another way to optimize your live queries is by getting rid of unnecessary oplog notifications, by moving a collection related to bulk updates into a different database, and manually connecting to it without the oplog. For optimizing your application startup loading time, you could use Lighthouse, an open-source, automated tool for improving the quality of web pages. It allows for auditing performance and accessibility, and once it assesses your web app’s performance, it will give you instructions on how to handle the detected issues.
Create MongoDB indexes to boost the performance of your app
MongoDB's indexes - special data structures storing a small portion of the collection’s data set - are extremely important since they allow for creating easily findable fields. When we have large data collections, indexes make all the difference and significantly improve the performance of our app. To create indexes, you have to simply use createIndex option on your collection from the level of MongoDB Shell, or _ensureIndex from the level of the Meteor app code, or manually in MongoDB Atlas.
Whenever you perform a document-searching query using your created index, MongoDB will check the index first, rather than scanning the entire collection. That is exactly how you save time and optimize your app’s performance!
What does it mean to scale your Meteor project?
They are different ways of scaling, mainly vertical scaling and horizontal scaling. Vertical scaling boils down to adding more resources (CPU/RAM/disk) to your server, while horizontal scaling refers to adding more machines or containers to your pool of resources. Horizontal scaling for Meteor projects normally includes either running multiple instances of our app on a single server with multiple cores or running multiple instances on multiple servers.
How to scale your Meteor project?
Let us focus on horizontal scaling, which is a popular choice for many Meteor projects, as it allows for reducing costs of commodity hardware. To scale horizontally, you need to add extra CPU cores/nodes/containers and strengthen your network by adding load balancers. Sometimes you need to scale either your database or your application, but there are times when it is necessary to scale both. The process of scaling MongoDB is similar: increasing your CPU, RAM, and drive capacity.
Since optimization is the most important part of any scaling process, remember to optimize your non-Meteor code, too. This may include optimizing your GraphQL resolvers in GraphQL API. Another solution would be parallelizing business logic to maximize efficiency.
What are the potential problems that you might encounter when you try to scale?
1. Heavy pub/sub (DDP) usage
Solution: make sure to identify the biggest resource drains to optimize your app. Consider disabling reactivity for certain parts of the app.
2. An abundance of bandwidth-heavy tasks
Solution: move bandwidth-heavy tasks like image hosting and video hosting to a content delivery network (CDN) to ease the load on your server.
3. Overloaded MongoDB cluster
Solution: do not process everything on your server. Your MongoDB cluster can handle lots of your processing loads. It is possible thanks to MongoDB aggregations, which, for example, allow for calculating most of the basic statistics (e.g. max, min, average) fully in the database.
4. Functions are holding up your server processing
Solution: Prefer asynchronous code on the server to make sure no functions needlessly wait for other functions to continue with processing. Consider parallelization whenever possible. Whenever you can, transfer CPU-heavy tasks such as image compression and video encoding, to external micro-services.
5. Long and unnecessary processing caused by client actions
Solution: Minimize the number of timers driven by client actions. If you have many clients, using timing delay functions in your code will not work to your advantage.
All things considered, Meteor is very much scaling-friendly, and you can easily scale a Meteor app if you optimize the performance of your application and prepare it for greater traffic. It is easier and cheaper to spot and eliminate issues, which arise in the early days of your app, so continuously monitor your app’s performance.
After you have optimized your MongoDB cluster, eliminated the biggest resource drains, and eased the load on your server, you will be ready to continue your optimization and scaling adventure and move onto new levels. There are multiple other steps you can take, most likely specific to your project. Here at Vazco, we would love to support you through your scaling process. Drop us a line to discover a world of brand-new Meteor possibilities!