GET Performance testing of popular (and not so) CMS / Sudo Null IT News FREE

Greetings habrazhitel! When development FFCMS for my projects, I highlighted one of my principal goals As fasting speed and minimal consumption of equipment resources - however, all the values ​​can be estimated sole by comparison, which I volition try to do in this article.

In my last issue, some users asked virtually the exit of system performance and I decided to test various popular and young CMS as of October 18, 2014. I would like to remark - the system was tested immediately after its installing and the data can significantly change in unrivaled direction operating theater another, depending on the period of use of the system - accelerando the database, list of files and former conditions.

Foreword

Before conducting such examination, I wanted to look at the results of similar tests - after all, most possible they sustain already been carried out earlier me. However, to my disappointment, I could non find a single comprehensive test that would compare at least 3 operating theatre Thomas More systems according to many criteria (possibly I was looking seriously?). This is astonishing, because dozens (or even hundreds) of thousands of webmasters, programmers, and large companies use the popular CMS - is it actually that nobody is interested in the issue of consumed resources away the scheme?

Test Objects

For testing, I chose a number of currently popular systems, some commercial and free. 10 systems were selected, their names and versions used are listed below:

• Bitrix standart - 14.5.0
• WordPress 4.0
• Drupal 7.32
• Joomla! 3.3.6
• Umi.cms 2.9.7
• FFCMS 2.0.3
• InstantCMS 2.1.1
• KodiCMS 12.20.37 (master)
• NetCat Standart 5.4
• HostCMS. Freebee 6.1.6

For paid systems, demo versions were hand-picked, which can equal downloaded on the web site of the creators.

Testing methodology

In decree for the test answer to be homogeneous for all products, a essential motorcar was created on which a web waiter, database and otherwise necessary examination software were installed. As a stand were used:

• Apache 2.2.27;
• PHP 5.3.28;
• MySQL Community Server 5.5.38-log;
• PHP modules: Zend Engine v2.3.0 (no opcache, xcache, acp, eaccelerator);
• Apache benchmark Version 2.3.

* UPD: the 2d php 5.5 examine was also conducted with OPCache enabled, as critics requested in the comments - its result is proportional to the main test. The results are shown in the last section.

To each one CMS was installed in the virtual surround of the stand, after which several lines were added to its write in code (almost frequently index.php), which made information technology possible to save the resultant role about the execution time of the script and the amount of memory used:

          $_test_loadstart = microtime(              true            );                          // код CMS                        $_test_loadend = microtime(              faithful            ); $_test_loadtime = number_format($_test_loadend - $_test_loadstart,                          3            ); $_test_memory = number_format(memory_get_usage()/              1048576            ,                          3            ); file_put_contents($_SERVER[              'HTTP_HOST'            ] .                          '.txt'            , $_test_loadtime .                          ';'                        . $_test_memory .                          "\n"            , FILE_APPEND);                  

Atomic number 3 a result, the sphere.local.txt file was received, which contained data along the script loading time and RAM consumption.
To produce requests to the site, ab (Apache benchmark) was used without creating parallel requests - the goal was non examination apache, the goal was to get average results for all CMS:

          ab -n 1000 http://sphere.local/ > ~/bench/ab/land.local.txt                  

Methodology for evaluating the results

In gross, the testing engineering was evaluated based on the above delineated technology in the following sections:

• Descriptive rating (Basic) - distribution sizing, number of files, size and characteristics of the database
• Approximate script download speed (phpload_time)
• Estimated RAM size (phpload_memory)
• Evaluating Apache bench mark run results (apache_benchmark)

In each testing section, for each CMS, grades (-2 <= x <= +2) were set contingent the results obtained (physical) and average (avg) of the entire sample:

• -2 - the worst result in this test
• -1 - the result is worse than the average therein test
• +0.5 - the result is close to the moderate within standart devination
• +1 - the effect is better than the average in that test
• +2 - the foremost result in this test

Based happening the test results all told categories according to all criteria, the resulting table was compiled, which shows the total results for all tests.
* I would besides same to linger over 1 point - UMI.CMS during examination showed outrageous results in download speed, therefore its results were excluded from the calculation of AVG in some testing categories.

Examination

Since we orientated ourselves with the material and testing methodology, we bequeath proceed directly to the examination itself. On a lower floor are the final results for each of the categories of testing.

Trial results in a convenient format: Google spreadsheet
Results for PHP 5.5 with OPCache (proportional to the main ones): Google spreadsheet .

Descriptive Assessment

Test / CMS	Bitrix	Wordpress	Drupal	Joomla	Umi.CMS	Ffms	Instantcms	KodiCMS	Netcat	Hostcms	Avg	MIN	MAX
Installed size of it (MB)	191	18.5	11.7	20.1	105	18.9	11.9	23,4	98.4	17.5	51.64	11.7	191
File count	29908	1175	1074	5267	7128	1682	1650	3009	6391	2919	6020.3	1074	29908
Mysql table count	241	11	74	68	97	22	84	33	155	114	89.9	cardinal	241
Mysql table rows	21918	126	1421	461	14616	187	12587	46	31467	20302	10313.1	46	31467
Mysql database size (Megabyte)	eleven	0.56	4.6	3.9	6.2	0.356	6.8	0.948	2.7	1.9	3.8964	0.356	eleven
SUMMARY POINTS	-9	6	basketball team	5.5	-five	6	unity	6	-four	one

Estimation of script loading speed

Test / CMS	Bitrix	Wordpress	Drupal	Joomla	Umi.CMS	Ffms	Instantcms	KodiCMS	Netcat	Hostcms	Avg	MIN	MAX
AVG loadtime (sec)	0.50	0.497733	0.210623	0.325381	2.99	0,090089	0.548239	0.213725	0.3279	0.70523	0.38	0.09	0.71
MIN loadtime (sec)	0.45	0.467	0.19	0,305	2.9	0,083	0.518	0.194	0,304	0.144	0.30	0.08	0.52
MAX loadtime (sec)	0.69	0.661	0.317	0.489	3,218	0.135	0.851	1,387	2,076	0.91	0.83	0.14	2.08
SUMMARY POINTS	-one	-one	3	2.5	-6	6	-2.5	ace	-four	-2

Estimated RAM Size

Test / CMS	Bitrix	Wordpress	Drupal	Joomla	Umi.CMS	Ffms	Instantcms	KodiCMS	Netcat	Hostcms	Avg	MIN	Soap
AVG memory (mb)	28,944115	22,992	18,144003	12,556	23,089791	3,181	5,953808	6,485019	13,696	10,789239	14.5830975	3,181	28,944115
MIN memory (mb)	28,942	22,992	18,144	12,556	22,784	3,181	5,951	6,485	13,696	4,213	13.8944	3,181	28,942
Liquid ecstasy retention (mb)	29,177	22,992	18,147	12,556	23,095	3,181	6,885	6,504	13,696	10,848	14.7081	3,181	29,177
SUMMARY POINTS	-6	-3	-3	3	-3	6	3	3	3	-one

Apache benchmarks score

Examination / CMS	Bitrix	Wordpress	Drupal	Joomla	Umi.CMS	Ffms	Instantcms	KodiCMS	Netcat	Hostcms	Avg	MIN	MAX
Meter taken for tests (sec)	527,433	520,498	230.91	340,219	13 032.86	97,178	555,886	224,176	343,965	727,217	396.3868889	97,178	727,217
Complete requests	1000	1000	1000	1000	1000	1000	1000	1000	1000	1000	1000	1000	1000
Failing requests	965	0	0	0	697	0	0	0	0	955	213,3333333	0	965
Requests per second (# / Securities and Exchange Commission)	1.9	1.92	4.33	2.94	0.08	10.29	1.8	4.46	2.91	1.38	3,547777778	1.38	10.29
Time per request (ms)	527,433	520,498	230.91	340,219	13032,857	97,178	555,886	224,176	343,965	727,217	396.3868889	97,178	727,217
Channel rate	69.51	14.9	32,99	28.61	2.28	105.48	92.36	13.33	31.74	23.98	45,87777778	13.33	105.48
Sum-up POINTS	-quatern	-2	four	2	-12	ten	0	four	2	-6

Final results

Test / CMS	Bitrix	Wordpress	Drupal	Joomla	Umi.CMS	Ffms	Instantcms	KodiCMS	Netcat	Hostcms
Basic	-9	6	five	5.5	-five	6	one	6	-quatern	one
phpload_time	-one	-one	3	2.5	-6	6	-2.5	indefinite	-four	-2
phpload_memory	-6	-3	-3	3	-3	6	3	3	3	-one
Apache benchmark	-four	-2	quartet	2	-12	decade	0	four	2	-6
SUMMARY RESULT	-20	0	+9	+13	-26	+28	+1.5	+14	-3	-cardinal

Test Results

At once information technology's time to kayo the results of our testing and in brief evaluate the result. All our results are settled from 0 to 2 sides: those that got a negative result I would not recommend anyone to consumption at completely if you value computer science resources (this is just my personal feeling - I understand dead that the functionality of such systems can make up an magnitude higher than others).
One of the important test results was the fact that not a single open system received a negative rating , and all commercial products were behind this line (+6 / -4 in favor of harsh germ).
I wouldn't like to draw other conclusions; I'll just give you the final graphs of important tests, draw your possess conclusions (all pictures are clickable):

Afterword

By no means did I want to bruise anyone - I only provided the results of subjective testing of several CMS. You always ingest the right to decide whether resource consumption is of bang-up grandness to you, perhaps more the gadget of development / managing a site.

Results in a accessible format: Google spreadsheet
Results for PHP 5.5 with OPCache (proportional to the principal ones): Google spreadsheet
Testing in plain-text: Pick from Yandex.Disk

DOWNLOAD HERE

GET Performance testing of popular (and not so) CMS / Sudo Null IT News FREE

Posted by: gordoncomanny.blogspot.com

Share this post