Recently, a prospective Nuxeo client, while evaluating document management software, asked us to demonstrate Nuxeo system performance and scalability capabilities in a proof-of-concept experiment. His application deals with a massive number of documents, and requires rapid response times for a large number of simultaneous users. The goal of this proof-of-concept experiment was to see if Nuxeo was up to the task.
We were happy to jump into the ring–our engineers are keen sports for these sorts of activities. Now that the dust has settled, and the benchmarks are finished, we thought we would share the results. We have not taken the time to provide much additional information on the performance of the Nuxeo Platform, so this provides us with that opportunity. The results are in line with our other benchmarks, and more than ever confirm that Nuxeo is best-in-class when it comes to performance and scalability!
The Set Up
One of the goals of the experiment was to assess the performance of the system running on a complex architecture. This was not a simple Nuxeo installation, as is the case for most of our continuous performance testing, but rather a cluster setup. Our team decided to run the benchmarks on an architecture made of two nuxeo application servers and four repository servers. We also decided to run this architecture on a standard public cloud infrastructure, which we know is not optimum for response times and scalability. In any event, high-end, on-premises hardware would give better numbers, but would also require a longer set up, and we only had a couple days for this project.
Here, we have to mention that Cloud Computing technology has changed the way we execute this kind of project. We make the simple, efficient deployment of the Nuxeo Platform a priority, especially for our Nuxeo Cloud content management offering. Thanks to that, we had the knowledge and the tools to quickly deploy a complex architecture on the Cloud. The Cloud we used here (as well as for our Nuxeo Cloud offering) is Amazon Web Services (AWS), but of course this could be set up with other cloud providers.
We ended up with two Nuxeo servers running in cluster mode behind elastic load balancing, connected to four content repositories, each running on a dedicated PostgreSQL database. We stored the binary data on Amazon’s S3 blob service — a built-in feature for Nuxeo.
The Test
The first challenge of a test like this is to load the system with enough data. In our case, we weighed our Nuxeo installation down with 66 million documents. Each document consisted of a simple document object in the Nuxeo repository, with basic metadata, and an image that, for instance, would be the result of a capture process on a remote system, e.g. account statements, invoices, or other business documents.
The test scenario was a snap. The test requirements were actually a fairly straightforward use of the Nuxeo Document Management module. Concretely, the test consisted of a sequence of searches in the document repository, displaying the document, and downloading binary files attached in the browser. All this was to be done via the standard Nuxeo Document Management user interface, which is a JSF application. Here again, there is undoubtedly room for improvement. Running the same benchmark directly through the Nuxeo API would also be interesting, and by “interesting” we mean faster!
Performance aside, what we were really interested in assessing was stability. With that in mind, we took the user sequence described above and simply executed it for a multitude of simultaneous users over a long period; this activity on the application was generated by Funkload, our load testing tool of choice.
Funkload is home-brew software that has been developed over the years with Benoit Delbosc as the lead developer of the project. Like much of what Nuxeo does, Funkload has been developed in an open source manner. The software is open source and we’ve been inviting other people to contribute and use it as much as we do. Today, seven years after the first version, Funkload is a remarkable, high-quality piece of software with a very active community. While it has little to do directly with the content management solutions that Nuxeo brings to the market–which may be a reason why we don’t talk that much about Funkload–it is as much a part of the Nuxeo development infrastructure as other more well-known components. If at some point you need load testing software, whether to stress a Nuxeo application, or some other platform, we highly recommend Funkload for serious stress tests!
Results and Perks
Here is what the test told us: the platform consistently delivers 70 pages of content per second with 150 concurrent users harnessing the system, making requests every second, and downloading content. The platform is very stable on test runs of up to 5 hours. According to the Funkload reporting, the main limiting factor appears to simply be the I/O of the system–the hard drives.
In our opinion, 70 pages per second of full content access for 150 concurrent users on a complex cluster system, and with a repository loaded with 66 million documents is already a great outcome. Then, take into account that 150 concurrent users harnessing the platform every second is much higher than the typical usage of an ECM or document management platform. We can safely extrapolate this number to something like 1500 concurrent users in a real-world production environment.
No other content management system can replicate these kinds of results. Add to this the fact that it was done on virtual, cloud-based commodity servers, and you have the first takeaway:
Nuxeo and its Cloud capabilities are a major step forward, making Cloud Document Management a reality for an enterprise-grade project. Together with AWS or a comparable IaaS service, it is now possible to set up a dedicated complex architecture that delivers high-performing, highly scalable content and a document management solution for a fraction of the cost of yesterday’s solutions. The fact that it is cloud-based may take a backseat to the real story: this level of performance for the price constitutes a game-changer. Cloud computing is ready to run high-performance, heavy-load and content-centric applications, at least if they are running the Nuxeo Platform.
The second conclusion we draw from this benchmark is that these limits can be pushed much further. The architecture set up in question is totally scalable horizontally. Imagine for a moment that you want to add repositories or nodes to the Nuxeo cluster. Not a problem–the numbers increase linearly.
What about scaling vertically? Can we improve the intrinsic performance of the system? If we swap out our Medium AWS machines for better ones, we’ll see an improvement. Or, for that matter, let’s imagine we go even further, and move to dedicated hardware. We’ll examine that I/O limitation we encountered in our Funkload testing, and we decide to tune a high-end file storage back-end system. Imagine we fine tune the other components in the setup. What numbers would we get then? Twice as fast? Two-thirds? Honestly, we can’t say, but what matters is that we know there is a lot of room for improvement for critical large scale applications, and there are two aspects than can be leveraged: horizontal and vertical scalability.
This post is just a short summary of the benchmark test. We’ve been running other benchmarks as well, and we constantly monitor the performance of our platform to make sure it evolves in the right direction. Please contact us if you are interested in this topic, and want more technical insight.
The post Nuxeo Platform Performance and Scalability: Beyond Passing the Test appeared first on Nuxeo Blog.