2

I have here a custom made CMS (based upon MVC and made by somebody else) and it includes a massive amount of 200 files (896kb total source size and a peakmem usage of 7MB) on each request.

Memory usage is OK but it is quite large amount of files if you ask me.

For example, my framework includes only a maximum of 20 files (classes) to produce a page.

My question is: Does this effects performance when a large amount of users hit the site? Is it stressful?

I have tested it with this small shutdown script (put on top of index.php):

// Measure script, very useful
function __getBytesShortNotation( $iBytes, $iDecimals = 0 )
{
    $sResult   = @max( 0, $iBytes );
    $iKiloByte = 1024;
    $iTeraByte = $iKiloByte*$iKiloByte*$iKiloByte*$iKiloByte;
    $iGigaByte = $iKiloByte*$iKiloByte*$iKiloByte;
    $iMegaByte = $iKiloByte*$iKiloByte;

    if( $sResult < $iKiloByte )
     { $sResult.='B'; }
    elseif( $sResult >= $iTeraByte )
     { $sResult = round( $sResult / $iTeraByte, $iDecimals ).'Tb'; }
    elseif( $sResult >= $iGigaByte )
     { $sResult = round( $sResult / $iGigaByte, $iDecimals ).'Gb'; }
    elseif( $sResult >= $iMegaByte )
     { $sResult = round( $sResult / $iMegaByte, $iDecimals ).'Mb'; }
    elseif( $sResult >= $iKiloByte )
     { $sResult = round( $sResult / $iKiloByte, $iDecimals ).'Kb'; }

   return $sResult;
}

function __includetest__()
{
  // get REAL usage
 $iMem = memory_get_usage( true );
 $iPeakMem = memory_get_peak_usage( true );
 $aIncludes = get_included_files();

 $iSize = 0;
 $s = null;
 $i = $iCount = count( $aIncludes );

 while( $i-- )
  { $file = $aIncludes[$i];
    $iSize+= filesize( $file ); 
    echo $file.' filesize '.filesize( $file )."\n";
  }

 include( 'sysutils.php' ); 
 echo __getBytesShortNotation( $iSize )." - $iSize Bytes \n"; 
 echo 'files: '. $iCount."\n";
 echo 'Real used memory: '.__getBytesShortNotation( $iMem ).' '.$iMem.' Bytes'."\n";
 echo 'Real peak memory: '.__getBytesShortNotation( $iPeakMem ).' '.$iPeakMem.' Bytes'."\n";

}; 

register_shutdown_function( '__includetest__' );   

EDIT: Did some research I have extend the script a bit (not posted here) and do some test. I'm not at the office right now so i will post the results of the CMS later. In this test a heavy CMS like Wordpress and my own framework. When I compare Wordpress to this CMS, Wordpress uses 'only' 105 files max to include. Comparing those with my own framework is maybe a little unfair because the CMS is seperated but see clear differences in runtime MT. I have done some test with cache (APC) and without cache. Here the results:

* **Own framework - seperated CMS (APC Enabled)

Files    : 30 files in 7 unique directories
TotalSize: 1Mb - 1107807 Bytes 

Functions: 1535 (internal)
Functions: 522 (user) + 1535 (internal) = 2057
Classes  : 167

Runtime  : 0.26 MT
Real used memory: 4Mb - 4456448 Bytes
Real peak memory: 5Mb - 4718592 Bytes

* **Own framework - seperated CMS - Tested at the office (APC Enabled)

Files    : 28 files in 7 unique directories
TotalSize: 1Mb - 1080506 Bytes 

Functions: 1708 (internal)
Functions: 520 (user) + 1708 (internal) = 2228
Classes  : 177

Runtime  : 0,48 MT
Real used memory: 7Mb - 6815744 Bytes
Real peak memory: 7Mb - 6815744 Bytes

* **Own framework - seperated CMS (APC Disabled)

Files    : 30 files in 7 unique directories
TotalSize: 1Mb - 1107807 Bytes 

Functions: 1516 (internal)
Functions: 522 (user) + 1516 (internal) = 2038
Classes  : 166

Runtime  : 0.33 MT
Real used memory: 9Mb - 9699328 Bytes
Real peak memory: 9Mb - 9699328 Bytes

* **Wordpress - Default Theme - no plugins activated (APC enabled)

Files    : 83 files in 6 unique directories
TotalSize: 3Mb - 2713670 Bytes 

Functions: 1535 (internal)
Functions: 2092 (user) + 1535 (internal) = 3627
Classes  : 200

Runtime  : 1.26 MT
Real used memory: 10Mb - 10485760 Bytes
Real peak memory: 10Mb - 10485760 Bytes

* **Wordpress Admin - no plugins activated (APC enabled)

Files    : 105 files in 8 unique directories
TotalSize: 3Mb - 3341902 Bytes 

Functions: 1535 (internal)
Functions: 2464 (user) + 1535 (internal) = 3999
Classes  : 207

Runtime  : 1.17 MT
Real used memory: 7Mb - 7077888 Bytes
Real peak memory: 7Mb - 7340032 Bytes

* **Wordpress - Default Theme - no plugins activated (APC disabled)

Files    : 83 files in 6 unique directories
TotalSize: 3Mb - 2713674 Bytes 

Functions: 1516 (internal)
Functions: 2092 (user) + 1516 (internal) = 3608
Classes  : 199

Runtime  : 1.22 MT
Real used memory: 12Mb - 12845056 Bytes
Real peak memory: 12Mb - 12845056 Bytes

* **Wordpress Admin - no plugins activated (APC disabled)

Files    : 105 files in 8 unique directories
TotalSize: 3Mb - 3341902 Bytes 

Functions: 1516 (internal)
Functions: 2464 (user) + 1516 (internal) = 3980
Classes  : 206

Runtime  : 1.3 MT
Real used memory: 16Mb - 16777216 Bytes
Real peak memory: 16Mb - 17039360 Bytes

--- be continued --

Update 2 - at the office

* *The MVC CMS - first run *

Files    : 199 files in 82 unique directories
TotalSize: 1Mb - 1241778 Bytes 

Functions: 1708 (internal)
Functions: 1 (user) + 1708 (internal) = 1709
Classes  : 295

Runtime  : 12.55 MT
Real used memory: 6Mb - 6029312 Bytes
Real peak memory: 6Mb - 6029312 Bytes

* **The MVC CMS - second test

Files    : 199 files in 82 unique directories
TotalSize: 1Mb - 1241778 Bytes 

Functions: 1708 (internal)
Functions: 1 (user) + 1708 (internal) = 1709
Classes  : 295

Runtime  : 1.31 MT
Real used memory: 5Mb - 5242880 Bytes
Real peak memory: 5Mb - 5242880 Bytes

** ** END OF TESTS **

MVC CMS Some first impressions:

  • Lot of (litte) files 199!
  • Lot of directories 82!
  • Slow first runtime
  • Huge memory use on first run
  • Less global functions (which is not bad)

So what could be a bottleneck, files or directories or something else?

Paul Roub
  • 36,322
  • 27
  • 84
  • 93
Codebeat
  • 6,501
  • 6
  • 57
  • 99
  • 2
    Do the files get auto-loaded when they are needed or does it include all 200 for every request? – naththedeveloper Feb 05 '14 at 16:56
  • 2
    It sounds like a lot, but more importantly, do you have a performance problem? If you are looking to grow your userbase, how does your site perform under, say, 30% increased load on the same hardware? If your CPU is not under stress and your disks aren't a bottleneck, don't worry about it. Remember that premature optimisation is the root of all evil `;-)`. – halfer Feb 05 '14 at 16:57
  • 1) The files are autoload (Kohana framework involved) but because they are required by design, the files will be loaded each time ;-). 2) These kind of Self-Made CMS runs locally in different configurations for different clients and yes there are some performance problems so we want to figure out what can cause the problem. I have figure out already that 200 files is insane. – Codebeat Feb 05 '14 at 22:06
  • NOTE: And 200 files without the CMS, just the index page (HTML output). – Codebeat Feb 05 '14 at 22:27
  • First and second run difference is too big, so files are probably not the case. Last time I've seen one of those, it was caused by loading a lot of data from external source and caching it in files. I'd recon xhprof profiling, graph will show the exact bottleneck. – lchachurski Feb 06 '14 at 17:38
  • Hai veNuker, thanks for the reply. I will try the xhprof some day. Probably has it something to do with caching. When not in cache 200 files is much and when in cache there is no problem. So running the CMS on a domain without caching (APC for example) is like to be a real nightmare, with caching however, 200 files is only a problem when the files are not cached in memory. That explains the high memory peak and the slow runtime. – Codebeat Feb 07 '14 at 11:19

1 Answers1

1

Of course it is, every include_once or require_once have time, memory usage and i/o impact on the server. Frameworks like Symfony lower amount of includes by combining classes to just a few files.

You could use xhprof to see how much time is lost for includes. To see server impact, try to run apache ab and monitor server with tool like htop.

[edit]

Fast tweaks - use APC for PHP < 5.4 or Opcache for 5.4 and higher.

lchachurski
  • 1,770
  • 16
  • 21
  • Ah thanks! Will take a look at this (I know APC already and use it). – Codebeat Feb 05 '14 at 21:58
  • As far as Symfony goes each class has it´s own file as per PSR-1. So do most other frameworks. Dunno where you got the idea that most frameworks combine classes to limit the amount of files included. – user555 Feb 06 '14 at 15:43
  • Yes you're right about PSR-1, but while execution those classes are merged, in Symfony case see - app\cache\dev\classes.php – lchachurski Feb 06 '14 at 17:12
  • In addition actual implementation, you will find plenty "addClassesToCompile()" usage across the framework https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpKernel/DependencyInjection/Extension.php – lchachurski Feb 06 '14 at 17:25