Runkit 1.0.4: PHP5 (up to 5.6+), Closures, and 12 New Features

Runkit 1.0.4 for PHP has been released!

Congratulations on the new long-awaited mega-release to all Runkit users! If you are regularly using Runkit and are familiar with its features, history and evolution, then you can jump directly to the description of changes of release 1.0.4. In any case, I suggest reading the full article.

What is Runkit?

Runkit is a PHP extension that allows doing things impossible from the standpoint of the language. The extension’s functionally consists of three parts.

Runtime Manipulations

The first and largest part of Runkit's functionality allows to copy, modify, and delete dynamically (during running of a PHP code) such constructs that can't be changed via built-in facilities of PHP language.
Runkit can copy, redefine and delete existing functions (including internal ones), turn a class into a child-class inheriting parent's content (runkit_class_adopt()) or detach a class from its parent with removal of inherited content (runkit_class_emancipate()). With Runkit you also can add, copy, and delete methods of existing classes, add and delete their properties. Likewise, Runkit allows you to override and delete previously defined constants.

Runkit_Sandbox

The second principal part of the functionality is “sandboxes” (Runkit_Sandbox class). Using them you can run PHP code in an isolated environment. Each "sandbox" can be configured with its own PHP security options such as safe_mode, safe_mode_gid, safe_mode_include_dir, open_basedir, allow_url_fopen, disable_functions, disable_classes. In addition, each "sandbox" can have individual values for Runkit's INI-settings: own superglobals (will be described below) and prohibition of overriding built-in functions.
"Sandboxes" can load PHP files (via include(), include_once(), require(), and require_once()), call inside functions, execute arbitrary PHP code and print contained variables values. In addition, you can specify a function to capture and process the output of a "sandbox".
Within a "sandbox" you can create an object of an "anti-sandbox” class Runkit_Sandbox_Parent which connects a "sandbox" with its parent environment. Functionality of an "anti-sandbox" is very similar to the functionality of a "sandbox", but for security reasons each type of communications with the outside environment should be explicitly enabled during "sandbox" creation.

Superglobals

Runkit also lets you add new PHP superglobals. To add these variables, you just list their names separated by commas in the property runkit.superglobal in the PHP configuration file.

Other

In addition to the three main parts of functionality Runkit also provides tools for PHP-code syntax checking (runkit_lint and runkit_lint_file) and runkit_import() function which allows to import a file as the include() does, but runkit_import() ignores the entire global code of this file. Depending on the flags runkit_import() can import a function or class (fully or partially) with redefining or preserving existing ones.

Why Runkit is necessary?

Runkit helps PHP-programmers in doing a lot of different things. I'll tell you about a few most common.

Patching other people's programs

Imagine that you use a third-party library (or a framework) and at some point you need to change its behavior. However, the code to be changed is in a private-method of some class in this library. The obvious solution is to edit a file that contains the method. It is a working solution, but the code of the library is now changed and updating becomes troublesome task, because of the necessity of applying the patch every time you update the library. Another solution is to override a method by using Runkit. It may be done via a call of function runkit_method_redefine(). There are also similar solutions for overriding existing program functions (runkit_function_redefine()) and for overriding existing constants (runkit_constant_redefine()). Such modifications of program code at running time are named “monkey patching”. At specialized Internet forums you can find a variety of recipes of patching such libraries as WordPress, 1C-Bitrix, CodeIngniter, Laravel etc by using Runkit. For solving some problems it may be useful to replace built-in PHP functions and Runkit gives you this ability too.

Running User Code in Isolated Environment

Runkit_Sandbox is often used for making the environment for execution of user code. Being properly configured it enables people to isolate code from the main system. In the simplest form it looks like this:

$options = […];
$sandbox = new Runkit_Sandbox($options);
$sandbox->ini_set(…);
$sandbox->eval($code);

Other uses

With runkit you can also organize updating of the program code on the fly, for example, in the same way as phpdaemon does (see https://translate.google.ru/translate?sl=ru&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fhabrahabr.ru%2Fpost%2F104811%2F&edit-text=&act=url).

Unit-Testing

Runkit's features for redefining of functions and methods make it extremely useful for writing unit-tests. With Runkit, creation of the stubs or “spies” during execution of tests becomes a simple matter, even if the architecture of the code under test does not support dependency injections. There are ready-made libraries that implement methods and functions substitution with stubs in the context of unit-tests (for instance, ytest, phpspy and others). With proper selection of the library you can get amazingly simple tests (see, for example, here).

The History of Runkit

Beginning

Runkit has been created in 2005 by Sara Golemon. The last version authored by the founder (v. 0.9) was released on 06.06.06. In October 2006 Sara has ceased to support the extension and the version 1.0 has not been released. At that time Runkit has contained the functions for manipulating constants, functions, methods. There also have been the runkit_import(), sandboxes, superglobals, functions for syntax checking, and a function for adding class properties. Online documentation on php.net (http://php.net/runkit) has been frozen near version 0.7, so actually even the part of functions made by Sara herself are not described there by now. In addition, almost all of Runkit's functionality in this documentation is labeled as experimental. That was true in 2006, but not today.

Decay

From October 2006 till October 2009, the extension was not supported by anyone, but PHP language went ahead, that is why, despite the patches made by some of community members, Runkit was unstable, causing segmentation faults even with PHP 5.2.

Rebirth

In October 2009, I began to mend Runkit and then to develop it at https://github.com/zenovich/runkit. I'll tell you about the releases issued during this time and changes they include.

Release 1.0.0 (April 1, 2010)

In fact, it has never been released; it's just a fake :). All the changes made by the community members after the release of version 0.9 and before the release of 1.0.1 are included in this version.

Release 1.0.1 (October 3, 2010)

It was the first real release Runkit after 2006. Now Runkit supports all versions of PHP prior to 5.3 inclusively. More than ten serious errors have been eliminated including some causes of segmentation faults. The key ones:

• segmentation fault on importing properties and constants with the constant-array values,
• crash on importing functions and methods having static variables,
• crash on manipulating functions
• segmentation fault caused by runkit_method_copy() when dealing with protected methods,
• segmentation fault on PHP's shutdown after changing built-in functions,
• crash on calling the original method after applying runkit_method_copy() to it, if the method had static variables
• names created by Runkit's methods are no longer converted to lowercase.

The release 1.0.1 adds an ability to define and modify the static methods using the new constant RUNKIT_ACC_STATIC:

runkit_method_add('MyClass', 'newMethod', '$arg1, $arg2',
                  '/* some code here */', RUNKIT_ACC_STATIC);

An ability to import static properties into the classes was also added. When you import a class static properties will be copied by default, but their importing can be disabled using a new constant RUNKIT_IMPORT_CLASS_STATIC_PROPS:

// imports classes entirely
runkit_import('myfile.inc', RUNKIT_IMPORT_CLASSES);

// imports classes, but not imports their static properties
runkit_import('myfile.inc', RUNKIT_IMPORT_CLASSES & ~RUNKIT_IMPORT_CLASS_STATIC_PROPS);   

// imports only the static properties of classes
runkit_import('myfile.inc', RUNKIT_IMPORT_CLASS_STATIC_PROPS);

In addition, the release adds the ability to apply the closures to the sandbox using the Runkit_Sandbox::call_user_func() method.

Release 1.0.2 (October 5, 2010)

It was a bug-fix release. Compatibility with PHP 5.3 was improved.

Release 1.0.3 (January 2, 2012)

Inheritance on renaming methods via runkit_method_rename() was corrected. Building on Windows was repaired.

Release 1.0.4 (September 25, 2015)

The long-awaited mega-Release! Full support of PHP5 (up to PHP 5.6 inclusive) was introduced.
Plenty of work has been done to finally stabilize Runkit: For every change tests were run for each PHP version in four variants: with ZTS and without, under valgrind and without it. New tests were added for almost each change. This allowed identifying and fixing a lot of errors.
Among the important corrections I want to highlight fixes of the following errors:

• segmentation fault on changing, deleting, or renaming functions, methods, or properties of the classes for which Reflection objects have been previously created,
• segmentation fault on creating Runkit_Sandbox if the register_globals INI-setting has been switched on,
• crash on a syntax error in the file loading via runkit_import(),
• segmentation fault on manipulating constants with single-character names,
• crash on calling renamed private or protected method.

More than forty (!!!) important fixes were made in this release, the full list of them can be found in the package.xml file.
I'll tell about the major functionality changes below.

Functions and Methods

Closures

In PHP 5.3+ functions runkit_function_add(), runkit_function_redefine(), runkit_method_add(), and runkit_method_redefine() now support closures as parameters. For example, to override a function earlier you had to use an expression similar to

runkit_function_redefine('sprintf', '$s', 'return $s;');

which used the eval() to convert a given string into a byte-code and for that reason was very slow. Now it is possible to write

runkit_function_redefine('sprintf', function ($s) {return $s;});

There is no eval() anymore! Furthermore supporting this code is much easier since there are no more program parts inside of string literals! The same is applicable to functions runkit_function_add(), runkit_method_add(), and runkit_method_redefine().

Magic Methods

Also Runkit now fully supports manipulations with magic methods __get(), __set(), __isset(), __unset(), __clone(), __call(), __callStatic(), serialize(), unserialize(), __debugInfo() and __toString(). The same applies to constructors and destructors both in contemporary naming style, and in PHP4 naming style.

Doc Comments

Now, when you add or override methods or functions using the old syntax (when arguments of the new function and its body are passed as string literals) you can also specify the doc comments. To this end the functions runkit_function_add(), runkit_function_redefine(), runkit_method_add(), and runkit_method_redefine() got a new optional (the last in order) argument named doc_comment:

// overrides private method and sets a doc comment
runkit_method_redefine('MyClass', 'myMethod', '$arg', 'return $arg',
                       RUNKIT_ACC_PRIVATE, 'my doc_comment');                 

// adds a private method with a doc comment
runkit_method_add('MyClass', 'myMethod2', '$arg', 'return $arg', NULL, 'my doc_comment2');

When defining the functions and methods in the new style (via closures) doc comments can be set in the same way as on defining usual functions - via specifying a doc comment before the function's body. Both methods can be combined, and in this case a doc comment in a parameter gets a highest priority. In addition, the doc-comments are now properly being set during inheritance, copying, and renaming of methods or functions.

Returning Values ​​by Reference

An ability to add or override a function or a method so that a new function (or method) returns a reference was introduced. If the new function is being defined using the old syntax (when the body and arguments of the new function is passed as string literals), then for returning reference, you should also pass a new parameter (return_ref) equal to TRUE to the runkit_function_add() (or runkit_function_redefine()) function. For example,

// returns a value by reference
runkit_function_redefine('my_function', '$a', 'return $a;', TRUE);

To add or redefine a method via the old syntax you may use the 'flags' parameter with switched on RUNKIT_ACC_RETURN_REFERENCE bit in order to return a value by reference. For example,

// a protected method returning by reference
runkit_method_redefine('MyClass', 'myMethod', '$a', 'return $a;',
                       RUNKIT_ACC_PROTECTED | RUNKIT_ACC_RETURN_REFERENCE);

If you define a function or method with the new syntax (via closures), then all things are much easier: You need just to add an ampersand before the list of function's arguments:

// returns a value by reference
runkit_function_redefine('my_function', function &($a) {return $a;});

Properties of Classes

The internal implementation of the manipulation of the properties of the classes has been completely reworked. Adding, deleting or importing class properties now correctly affect descendant classes. Moreover, now these operations can affect existing objects of the class and its descendants. To enable this effect you need to switch on the RUNKIT_OVERRIDE_OBJECTS bit in the 'flags' parameter of functions runkit_default_property_add() or runkit_default_property_redefine(). For example,

// does not affect existing objects of the class and its descendants
runkit_default_property_add('MyClass', 'newProperty', 'value');

/* adds properties not only to the class and its descendants,
   but also to their existing objects */
runkit_default_property_add('MyClass', 'newProperty', 'value', RUNKIT_OVERRIDE_OBJECTS);

The same for the importing of class properties:

/* imports the class properties not overriding existing properties
   and without affecting existing objects */
runkit_import('myfile.inc', RUNKIT_IMPORT_CLASS_PROPS);

/* imports the properties of classes, overriding the existing properties,
   but without affecting existing objects */
runkit_import('myfile.inc', RUNKIT_IMPORT_CLASS_PROPS | RUNKIT_IMPORT_OVERRIDE);

/* imports the properties of classes, overriding the existing properties
   and changing the properties of existing objects */
runkit_import ('myfile.inc', RUNKIT_IMPORT_CLASS_PROPS |
                             RUNKIT_IMPORT_OVERRIDE | RUNKIT_OVERRIDE_OBJECTS);

In addition, a new function runkit_default_property_remove() has been introduced for removing class properties. If you want to remove a property not only out from the class and its descendants, but also out from their existing objects, you should pass an optional parameter (the last one in order) of the function runkit_default_property_remove():

// removes the property out from the class, but leaves it in existing objects
runkit_default_property_remove('MyClass', 'myProperty');

// removes the property out from the class and its existing objects
runkit_default_property_remove('MyClass', 'myProperty', TRUE);

Also now on adding or overriding the class properties you can pass not only scalar values, but also non-scalar ones.

Classes

Previously though the functions runkit_class_adopt() and runkit_class_emancipate() have been used to change the contents of classes, but they have not affected on class hierarchy (that is, after applying the runkit_class_adopt() function to a class the latter formally still did not have a parent, and after applying the runkit_class_emancipate() function the class still had a parent). Now it's fixed.

Namespaces and Letter Case of the Names of Entities

Manipulations with constants, functions, methods, and properties now fully support namespaces. Also Runkit no longer converts to lowercase names of properties, classes, methods and functions that it creates, as it did before.

Additional Security of Sandboxes

You can now switch off the allow_url_include INI-setting for Runkit_Sandbox'es. Also, regardless of the platform, the open_basedir INI-setting can be set by a list of paths (previously it was possible to specify only one path).

Updates

Installing and upgrading Runkit have become much easier. Now you can do it through the official channel pecl.php.net in a convenient manner familiar to all PECL users. Use this command to install the latest release of Runkit:

pecl install runkit

and this one for upgrading:

pecl upgrade runkit

In addition, archives for all releases are now available at http://pecl.php.net/runkit.

Conclusion

Today Runkit is used by many well-known companies and projects around the world for unit-testing and many other tasks. I am sure that Runkit is marked for a great future. This will be possible with your donations, which now can be easily made on the project's page https://github.com/zenovich/runkit or directly on a phpinfo() page with Runkit's settings.
Also if you just read this article don't hesitate to donate to Runkit right here.



Thank you!