Can Magento 2 Manage 100K Products effectively?
When we talk about the Magento 2 eCommerce platform, then a question that always arises on the top is “Can Magento 2 Manage 100K Products effectively?“
Surely, in return every seller or store owner with Magento 2 eCommerce store wants to hear the answer YES. So keeping the suspense apart, Magento 2 can really handle 100k products.
Not just 100k products the Magento 2 can also hold around 1 million products at once. So, let’s move forward to get the detailed information regarding the management and tech stacks of how this actually happens.
It will impact some of the aspects or elements of the store. Some of them are mentioned below:
The management of the products completely depends on which Magento version is in use. Of course the better will be the version, the best will be the product handling power.
It is of course that when more than 100k products will be loaded to the database then it may affect the search process in the frontend as well as backend.
The integration of the Elastic Search engine to your web store is one of the best options to be chosen. Elastic search is and highly scalable open-source full-text search engine.
This allows the store owner to easily search, store, and manage the heavy volume of products very quickly. So, get up and change your regular MySQL search engine to a complete and quick elastic search engine.
The interesting part is that the store owner can also offer their customers with amazing search types. Mentioning the interesting types below:
If the store owner is using this search in the store then the customer can search the product by the Name and SKU only.
With the Multiple search query, the customers can search the products by typing any of the p[roduct attributes provided by the admin in the backend. That may include the Name, SKU, Description, Short Description, etc.
With features like this, The elastic search engine will allow the customers to search among 100k products in no time. The store owner will also implement the Cron for Index Management for a daily, weekly, or monthly basis.
Cron setup will help to auto-replicate the changes to the Elastic search engine if any of the changes are done in the backend.
CSRF OR Cross-site request forgery is a web protection susceptibility that allows an attacker to force users to execute processes that are not supposed to be performed by them.
The impact of CSRF attacks is as dangerous as they can change the customer’s password, email, can access their account, can withdraw an amount, etc.
CSRF tokens can be used to protect the store from the CSRF Attacks. These tokens are secret, unique, and completely unpredictable and are attached to the HTTP request made by the client site. Later, when the same HTTP request is approached to the server site then it checks the token, if that token is not present then the request is rejected.
Since it is almost impossible for the attackers to create the HTTP request just similar to the victim user. Also, the attacker can not predict or construct the CSRF Tokens, or the parameters attached to the request.
Generate the Tokens -> Transmit the Tokens -> Validate the Tokens
So basically the tokens are generated using the PRNG i.e. pseudo-random number generator.
Now, throughout the lifecycle of the token, it is a must that they are also protected. So, they are transmitted to the client site by using the post method inside the HTML Form
Further, the request will be stored under the data of the user’s session and whenever the next request arrives then the system will first match that coming request with any token or not, and if the token is present then it will match it with the store’s token. If they are the same, the the request is passed.
Dissimilarity will lead to a complete rejection, and by this, the store sessions are tasks that can be reality managed and protected using CSRF.
As the tokens generated for each request will increase with the time and customer request, so one must have a proper storage system to store the tokens.
Redis can be the best in-memory data structure that is BSD licensed. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence.
After uploading the products one must check the disc storage consumed for such a good number of products. In PHP there is a sending limit is configured depending on some aspects, that are mentioned below:
The admin sets the product count limit for upload, which will be done with each installation.
The product size is also the considerable point on which the uploading process will depend. A server timeout is also set up so that the upload will be stopped after a particular time.
When the products are uploaded it is very important to take care of the following aspects, i.e
The systems must be compatible with and have the appropriate memory to upload such a huge amount of products.
Apart from all these some of them are already mentioned in the above article. By managing all these points one can make the installation process much easier.
Reindexing may take time as the number of products is pretty huge to upload, that is completely not an issue.
Not only the products, but we also need to manage the traffic when more than the expected customers start landing at your store. For that, one must use load balancers that will enhance the store user holding capacity.
With this, the client can access the required data from multiple servers. The load will not be forced into a particular system. AWS Elastic load balancer is suggested.
Surely, in return every seller or store owner with Magento 2 eCommerce store wants to hear the answer YES. So keeping the suspense apart, Magento 2 can really handle 100k products.
Not just 100k products the Magento 2 can also hold around 1 million products at once. So, let’s move forward to get the detailed information regarding the management and tech stacks of how this actually happens.
Impact On Multiple Elements
It will impact some of the aspects or elements of the store. Some of them are mentioned below:
- Product Attributes
- Categories including the depth of its tree
- Configurable/ Bundled products
- Customer Groups that have different product prices.
The management of the products completely depends on which Magento version is in use. Of course the better will be the version, the best will be the product handling power.
Make Search Simple Easy and Fast
It is of course that when more than 100k products will be loaded to the database then it may affect the search process in the frontend as well as backend.
The integration of the Elastic Search engine to your web store is one of the best options to be chosen. Elastic search is and highly scalable open-source full-text search engine.
This allows the store owner to easily search, store, and manage the heavy volume of products very quickly. So, get up and change your regular MySQL search engine to a complete and quick elastic search engine.
The interesting part is that the store owner can also offer their customers with amazing search types. Mentioning the interesting types below:
Simple Search Query
If the store owner is using this search in the store then the customer can search the product by the Name and SKU only.
Multiple Search Query
With the Multiple search query, the customers can search the products by typing any of the p[roduct attributes provided by the admin in the backend. That may include the Name, SKU, Description, Short Description, etc.
With features like this, The elastic search engine will allow the customers to search among 100k products in no time. The store owner will also implement the Cron for Index Management for a daily, weekly, or monthly basis.
Cron setup will help to auto-replicate the changes to the Elastic search engine if any of the changes are done in the backend.
Increase Store Security Using CSRF Protection
CSRF OR Cross-site request forgery is a web protection susceptibility that allows an attacker to force users to execute processes that are not supposed to be performed by them.
The impact of CSRF attacks is as dangerous as they can change the customer’s password, email, can access their account, can withdraw an amount, etc.
How to Prevent CSRF Attacks?
CSRF tokens can be used to protect the store from the CSRF Attacks. These tokens are secret, unique, and completely unpredictable and are attached to the HTTP request made by the client site. Later, when the same HTTP request is approached to the server site then it checks the token, if that token is not present then the request is rejected.
Since it is almost impossible for the attackers to create the HTTP request just similar to the victim user. Also, the attacker can not predict or construct the CSRF Tokens, or the parameters attached to the request.
Generate the Tokens -> Transmit the Tokens -> Validate the Tokens
So basically the tokens are generated using the PRNG i.e. pseudo-random number generator.
Now, throughout the lifecycle of the token, it is a must that they are also protected. So, they are transmitted to the client site by using the post method inside the HTML Form
Further, the request will be stored under the data of the user’s session and whenever the next request arrives then the system will first match that coming request with any token or not, and if the token is present then it will match it with the store’s token. If they are the same, the the request is passed.
Dissimilarity will lead to a complete rejection, and by this, the store sessions are tasks that can be reality managed and protected using CSRF.
As the tokens generated for each request will increase with the time and customer request, so one must have a proper storage system to store the tokens.
Redis can be the best in-memory data structure that is BSD licensed. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence.
How Does Upload Happens?
After uploading the products one must check the disc storage consumed for such a good number of products. In PHP there is a sending limit is configured depending on some aspects, that are mentioned below:
Product Count
The admin sets the product count limit for upload, which will be done with each installation.
Product Size
The product size is also the considerable point on which the uploading process will depend. A server timeout is also set up so that the upload will be stopped after a particular time.
Memory and Hardware Requirement
When the products are uploaded it is very important to take care of the following aspects, i.e
- RAM
- Memory
- CPU
The systems must be compatible with and have the appropriate memory to upload such a huge amount of products.
How to Optimize Magento 2 Installation with 100k Products?
- Enable the Full Page cache
- Lazy Loading
- Varnish
- Leverage Browser Caching
- Optimized Images
- Production Mode Enabled
- Images Format can be .webp
- Number Of Products per page
Apart from all these some of them are already mentioned in the above article. By managing all these points one can make the installation process much easier.
Reindexing may take time as the number of products is pretty huge to upload, that is completely not an issue.
Manage Huge Traffic
Not only the products, but we also need to manage the traffic when more than the expected customers start landing at your store. For that, one must use load balancers that will enhance the store user holding capacity.
With this, the client can access the required data from multiple servers. The load will not be forced into a particular system. AWS Elastic load balancer is suggested.
Expand Your Digital Horizons With Us
Start a new project or take an existing one to the next level. Get in touch to start
small, scale-up, and go Agile.