An Auto Loader Using PHP Tokenizer

A.J. Brown — Wed, 03 Dec 2008 05:07:08 +0000

What is an Auto Loader?

In PHP, an auto loader is a set of code that's responsible for "loading" a class that is needed for some piece of code (if it has not already been loaded). Typically it uses some formula or pre-determined list to figure out what file needs to be included in order for the class to be defined. In most applications the filename is determined by examining the class name.

Why Auto Load instead of using includes?

Although adding some overhead to your application, using an autoloader instead of directly requiring files will increase the flexibility of your application. With an autoloader, you can move files around in your project as you please, without breaking dependent code. It's also much more convenient for the developer, who no longer needs to know the exact location (or version) of a needed file.

Using an autoloader instead of directly requiring files will help keep your code decoupled. This is especially useful for writing unit-testable code by allowing you to mock dependent objects, and prevent unnecessary code from being run.

Our Class File Mapping Package

The class file map factory is responsible for scanning the files of a specified directory, indexing the classes and interfaces contained within those files, and generating a ClassFileMap object to represent the index. Each ClassFileMap is then used by a ClassFileMapAutloader to lookup the needed class.

To load the files we want to index, we're using SPL'sRecursiveDirectoryIterator andRecursiveIteratorIterator. Explaining those classes is outside the scope of this article.

The auto loader I wrote uses thetokenizer'stoken_get_all, which returns an array of tokens within a given source string. Tokens describe the relevance of a sequence of characters in PHP source, such as `whitespace`, `class`, `function`, `bracket`, etc. Once our file has been tokenized, we just look for all `class` and `interface` tokens, and look at the next non-whitespace token to select the class name.

Class File Map Factory

/**
 * Class to handle generating the class file map, and loading of classes
 *
 * @author A.J. Brown
 * @package com.hypermuttlabs
 * @subpackage packaging
 */
abstract class ClassFileMapFactory
{
 
 /**
  * Generates a class file map instance for the specified class path
  *
  * @param string $sClassPath the path to analyze
  * @param string $sName      the name for this class file map
  * @return ClassFileMap
  */
 public static function generate( $sClassPath, $sName = null )
 {
  $aClassMap = self::_getClassFileMapArray( $sClassPath, true );
  $oClassfileMap =  new ClassFileMap( $sName );
  $oClassfileMap->setClassPath( $aClassMap );
  return $oClassfileMap;
 }
 
 /**
  * Generates a class file map for the specified directory
  *
  * @return array
  */
 private static function _getClassFileMapArray( $sDirectory, $bRecursive = true )
 {
 
  if ( !is_dir( $sDirectory ) )
  {
   throw new AppException(
    'The specified location is not a directory' );
  }
 
  if ( !is_readable( $sDirectory ) )
  {
   throw new AppException(
    'The path `' . $sDirectory . '` is not readable' );
  }
 
  $sPath = realpath( $sDirectory );
 
  $oFiles = new RecursiveIteratorIterator(
   new RecursiveDirectoryIterator( $sPath ),
   $bRecursive ? RecursiveIteratorIterator::SELF_FIRST : null
  );
 
  $sHiddenFiles = '/\/\.\w+/';
 
  //------------------------------------------
  // Load the list of files in the directory
  //------------------------------------------
  foreach( $oFiles as $sName => $aFile )
  {
   if( !preg_match( $sHiddenFiles, $sName ) && !$aFile->isDir() )
   {
    $oFile = $aFile->openFile();
 
    $sContents = null;
    while( !$oFile->eof() )
    {
     $sContents .= $oFile->fgets();
    }
 
    //----------------------------------------
    // Tokenize the source and grab the classes
    // and interfaces
    //----------------------------------------
    $aTokens = token_get_all( $sContents );
    $iNumtokens = count( $aTokens );
    for( $i = 0; $i < $iNumtokens; $i++ )
    {
 
     switch( $aTokens[ $i ][ 0 ]  )
     {
      case T_CLASS:
      case T_INTERFACE:
       $i += 2; //skip the whitespace token
       $aDeclarations[ $aTokens[ $i ][ 1 ] ] = $sName;
       break;
     }
    }
   }
  }
 
  return $aDeclarations;
 }
}

You'll notice we're using a regex to check the directory name of our files before processing them. This prevents hidden folders (such as .svn) from being processed, which will both reduce overhead, and prevent diff and history files from being scanned.

Class File Map Autoloader

The ClassFileMapAutoloader is used by PHP to locate classes which have not yet been defined. It must be added to the autoload stack usingspl_autoload_register by calling it's registerAutload() method. Once registered, it will use all attached ClassFileMaps to lookup class names and require their files.

/**
 * Autoloads classes using class file maps
 *
 * @author A.J. Brown
 * @package com.hypermuttlabs
 * @subpackage packaging
 *
 */
class ClassFileMapAutoloader
{
 private $_aClassFileMaps = array();
 
 /**
  * Adds a class file map for use by this autoloader.  ClassFileMaps are grouped
  * by their name if the second parameter is true, resulting in a second
  * class file with the same name overwriting the first.
  *
  * @param ClassFileMap $oClassFileMap
  * @param bool $bUseName use the value of {@link ClassFileMap::getName()}
  *  as the key
  *
  * @return void
  */
 public function addClassFileMap( ClassFileMap $oClassFileMap, $bUseName = true )
 {
  if( $bUseName )
  {
   $this->_aClassFileMaps[ $oClassFileMap->getName() ] = $oClassFileMap;
  }
  else
  {
   $this->_aClassFileMaps[] = $oClassFileMap;
  }
 }
 
 /**
  * Registers this class with the spl autloader stack.
  *
  * @return bool
  */
 public function registerAutoload()
 {
  return spl_autoload_register( array( &$this, 'autoload' ) );
 }
 
 /**
  * Autloads classes, if they can be found in the class file maps associated
  * with this autoloader.
  *
  * @param string $sClass
  * @return string the class name if found, otherwise false
  */
 public function autoload( $sClass )
 {
  if( class_exists( $sClass, false ) || interface_exists( $sClass ) )
  {
   return false;
  }
 
  $sPath = $this->_doLookup( $sClass );
  if ( $sPath !== null )
  {
   require_once $sPath;
  }
 
  return true;
 }
 
 /**
  * Loop through class files maps untill a match is found
  *
  * @param string $sClassName
  * @return string the path of the class, or null if not found
  */
 private function _doLookup( $sClassName )
 {
  foreach( $this->_aClassFileMaps as $oClassFileMap )
  {
   $sPath = $oClassFileMap->lookup( $sClassName );
   if ( !is_null( $sPath ) )
   return $sPath;
  }
 
  return null;
 }
}

Class File Map

The ClassFileMap represents an index of php files of a specific location, allowing the ClassFileMapAutoloader to determine a filename from a class name.

/**
 * Manages a map of class names to their paths
 *
 * @author A.J. Brown
 * @package com.hypermuttlabs
 * @subpackage packaging
 *
 */
class ClassFileMap
{
 
 /**
  * @var array
  */
 private $_aClassMap;
 
 /**
  * Stores a name for this class file map.  For user usage.
  * @var string
  */
 private $_sName;
 
 /**
  * Stores the timestamp of this class file map's creation.
  * @var int
  */
 private $_iCreated;
 
 /**
  * Constructor
  *
  * @param string $sName an optional name to give this class file map.
  */
 function __construct ( $sName = null )
 {
  $this->_iCreated = time();
  $this->_sName = $sName;
 }
 
 /**
  * Maps the specified class to the specified path. If the first argument
  * is an array, it is expected that the array contains name-value pairs
  * of class names and thier paths.
  *
  * @param string|array  $mClassName the class name to map
  * @param string   $sPath the path to map to
  *
  * @return void
  */
 public function setClassPath( $mClassName, $sPath = null )
 {
  if( is_array( $mClassName ) )
  {
   foreach( $mClassName as $sClassName => $sPath )
   {
    $this->setClassPath( $sClassName, $sPath );
   }
 
   return;
  }
 
  $this->_aClassMap[ $mClassName ] = $sPath;
 }
 
 /**
  * Retreives the name of this class
  *
  * @return string
  */
 public function getName(  )
 {
  return $this->_sName;
 }
 
 /**
  * Determine if a class is mapped
  *
  * @param string $sClassName
  * @return boolean true if the class is mapped
  */
 public function isMapped( $sClassName )
 {
  return !empty( $this->_aClassMap[ $sClassName ] );
 }
 
 /**
  * Returns the path of the file if mapped, otherwise null
  *
  * @param  string  $sClassName the class to lookup
  * @return  string the full path to the file containing the class
  */
 public function lookup( $sClassName )
 {
  if ( !$this->isMapped( $sClassName ) ) return null;
 
  return $this->_aClassMap[ $sClassName ];
 }
 
 /**
  * Returns an array of classes which exist within a given file path
  *
  * @param unknown_type $sFileName
  * @return unknown
  */
 public function getClassesInFile( $sFileName )
 {
  return array_keys( $this->_aClassMap, $sFileName );
 }
 
 /**
  * Retreives the entire class-file map as an array with class names as keys
  * and their paths as values
  *
  */
 public function getClassMap()
 {
                return $this->_aClassMap
 }
}

Usage Example

//build a class file map for our applications classes
$oAppClassFileMap = ClassFileMapFactory::generate('/path/to/class/files');
 
//instanciate a new auto loader
$this->_oAutoLoader = new ClassFileMapAutoloader();
 
//add the class file map to the autoloader
$this->_oAutoLoader->addClassFileMap( $oAppClassFileMap );
 
//register the autoloader
$this->_oAutoLoader->registerAutoload();

And that's all there is to it. Once this code has been executed in your application, no further requires will be needed. We do add a slight overhead to the application, but the increased flexibility and modularity is well worth it, especially for larger applications.

Feel free to use the autoloader in this article in your own projects.

A.J. Brown's Blog » Autoload