Category Archives: open source

Increase performance of Vagrant + VirtualBox synced folders

The standard synced folders set up in Vagrant uses the VirtualBox’s shared folders feature. Unfortunately, the performance of shared folders leaves a lot to be desired. There is a solution to this poor performance, which involves switching to an NFS-based solution.

This can be done simply by adding the nfs flag to the config.vm.synced_folder setting in your Vagrantfile:

config.vm.synced_folder '.', '/vagrant', nfs: true

You also need to make sure you are using Vagrant’s private_network networking option:

config.vm.network "private_network", ip: "192.168.56.101"

Once you have done this, you can vagrant up (if your VM is not currently running) or vagrant reload (if you VM is currently running) to get NFS up and running. When you do this for the first time, you’ll likely be prompted to enter your administrator’s password so that the NFS details can be saved to /etc/exports:

==> default: Preparing to edit /etc/exports. Administrator privileges will be required...
Password:

The performance increase you experience will vary, but on my 1.3 GHz Intel Core i5 MacBook Air, I experienced a 6x performance boost. I used wrk to benchmark the VM web server performance before and after:

Using VirtualBox shared folders:

$ wrk http://localhost:8000
Running 10s test @ http://localhost:8000
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.00us    0.00us   0.00us     nan%
    Req/Sec     4.11      3.22     9.00     55.56%
  10 requests in 10.04s, 1.22MB read
  Socket errors: connect 0, read 0, write 0, timeout 10
Requests/sec:      1.00
Transfer/sec:    124.20KB

Using NFS:

$ wrk http://localhost:8000
Running 10s test @ http://localhost:8000
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.38s   214.51ms   1.86s    63.24%
    Req/Sec     5.33      3.83    19.00     71.43%
  68 requests in 10.05s, 8.40MB read
Requests/sec:      6.76
Transfer/sec:    855.64KB

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Listing your Git commits from the last 24 hours

Useful aide-memoire when you are creating end of day status emails:

git log --author="`git config user.name`" --pretty=format:"%cd %h %s" --date=short --since="yesterday"

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Allow per-developer Vagrantfile customisations

If you are sharing a Vagrant file across a team, for example, for configuring a local development environment you’ll know that sometimes individual developers want their own VM tweaks. For example, they may have a machine with more memory that they’d like to dedicate to the VM. In order to support this, you can add a config file for custom changes:

Example Vagrantfile showing custom memory and CPU cap:

# -*- mode: ruby -*-
# vi: set ft=ruby :
require 'yaml'

if File.exists?('vagrant.yml')
  settings = YAML.load_file 'vagrant.yml'
end

memory_use    = defined?(settings['memory']) ? settings['memory'] : 1024
cpu_execution = defined?(settings['cpucap']) ? settings['cpucap'] : 50

Vagrant.configure(2) do |config|
  config.vm.box = "puphpet/centos65-x64"
  config.vm.network "private_network", ip: "192.168.56.101"

  config.vm.provider "virtualbox" do |vb|
    vb.memory = memory_use
    vb.customize ["modifyvm", :id, "--cpuexecutioncap", cpu_execution]
  end

  config.vm.provision "ansible" do |ansible|
    ansible.playbook = "provision/playbook.yml"
    ansible.become = true
    ansible.raw_arguments = Shellwords.shellsplit(ENV['ANSIBLE_ARGS']) if ENV['ANSIBLE_ARGS']
    ansible.compatibility_mode = "1.8"
  end
end

In this example, we’ve specified two configurable settings — memory_use and cpu_execution. Values will be read from the settings file, if it exists, otherwise default values are set (lines 9-10). Memory is specified in megabytes, and CPU execution is the maximum allowed percentage of the hosts CPU usage.

Example vagrant.yml file:

memory: 4096
cpucap: 75

Don’t forget to add the vagrant.yml file to your .*ignore file to make sure it isn’t committed to your source repo.

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

HTML5, character encodings and DOMDocument loadHTML and loadHTMLFile

Extreme HTML Verschachteling

Whilst working on a script for my GetProThemes app recently, I came across a problem with PHP’s loadHTML and loadHTMLFile methods.

The problem

I noticed that when using loadHTMLFile to parse an HTML document, the character encoding — UTF-8 in this case — was not being taken into consideration. Because of this, there was some mojibake after I extracted some content from the document. Here is an example of the problem:

$i18n_str = 'Iñtërnâtiônàlizætiøn';

$html = <<<EOS
<!doctype html>
<head>
  <meta charset="UTF-8">
  <title>html 5 document</title>
 </head>
 <body>
<h1 id="title">$i18n_str</h1>
</body>
</html>
EOS;

$dom = new DOMDocument();
$dom->loadHTML( $html );
echo $dom->getElementById( 'title' )->textContent;

// output: Iñtërnâtiônà lizætiøn

After some digging into the PHP source code, I discovered this function, along with loadHTML, uses Libxml for determining the character set of the HTML document automatically. It uses a function named htmlCheckEncoding for this purpose. What this function does is to look for a meta tag declaring the character set. Unfortunately, it only looks for the HTML4 style declaration:


<META http-equiv="Content-Type" content="text/html; charset=UTF-8">

This means that if your source document is HTML5, it will not pick up the newer meta tag declaration which has this form:


<meta charset="utf-8">

It seems that this glitch has been fixed in version 2.8.0 of Libxml, but if you are stuck with an older version then I have created a workaround.

The solution

I have created a drop-in replacements for the loadHTML/loadHTMLFile methods which will automatically convert an HTML5 character set declaration, if it exists, into an HTML4 character set declaration and thus allowing Libxml to parse the document correctly.

Fixing the above example is trivial:

require_once 'DOMDocumentCharset.php';

$i18n_str = 'Iñtërnâtiônàlizætiøn';

$html = <<<EOS
<!doctype html>
<head>
  <meta charset="UTF-8">
  <title>html 5 test</title>
 </head>
 <body>
<h1 id="title">$i18n_str</h1>
</body>
</html>
EOS;
		
$dom = new DOMDocumentCharset();
$dom->loadHTMLCharset( $html );
echo $dom->getElementById( 'title' )->textContent;

// output: Iñtërnâtiônàlizætiøn

So, the fix involves:

1. Including the DOMDocumentCharset class
2. Instantiating DOMDocumentCharset rather than DOMDocument
3. Calling the new loadHTMLCharset method

The class will only activate the workaround if the installed Libxml version is less than 2.8.0, so upgrading Libxml will not break this code.

The source code can be found on GitHub: dom-document-charset

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Compiling PHP extensions using Xcode 4.3 and above

Prior to version 4.3 of Xcode, Apple bundled a load of command line tools with it that are used to compile native binaries, including PHP extensions. However, as of version 4.3, these tools are not bundled by default. Therefore, if you require these tools you need to take one of two actions:

1. If you already have Xcode 4.3 installed, go into the Components tab of the Download preferences panel and make sure Command Line Tools are selected

Xcode CLI tools installation

OR

2. Download and install the command line tools as an independent package from either;

a. Apple (requires Apple ID)
b. GitHub

Once you have the command line tools installed, you are able to compile native binaries.

If you found this post useful, it would be great if you could Like my Facebook page using the button in the sidebar. Thanks!

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Finding function dependencies in your PHP applications

wide web

A recent post on reddit/PHP asked an interesting question: how can you determine the extension requirements of your PHP application in a programmatic way? And could you add such a check to a continuous integration environment to validate dependencies?

I’ve come up with a relative simple solution, albeit with some caveats (see below). The process requires two distinct stages:

  1. Gather a list of extensions and functions from a PHP environment
  2. Scan PHP source code for function calls and flag up any that are not either a) available in the PHP environment or b) user defined.

I’ve posted some code to Github that demonstrates how this can be done. You can find the source code here: https://github.com/glenscott/php-dependencies.

For stage 1, you must determine the functions that are defined in your PHP environment. This is done by running get-env-functions.php either on the command line, or from within your document root. This will create a config file in the directory defined by CONFIG_PATH. This config file will be used by the second script, scan-dependencies.php.

scan-dependencies.php will scan through PHP source code defined by SOURCE_PATH and use the configuration file generated previously. After it finishes scanning, it will list all function calls made that are not defined in either PHP itself, or within the source directory.

Example Run

Getting details of your PHP environment

$ php get-env-functions.php 
Found 1743 total functions in 61 extensions available in PHP.

Scanning source code for dependencies

$ php scan-dependencies.php 
Found 3 function calls and 1 object instantiations in your script.
Function ps_setgray not defined.

In this example, the function ps_setgray was called in a script but not defined anywhere.

Caveats

  • Your source code and its dependencies must lie under one directory — included/required files outside this directory are not scanned
  • As it stands, only _function_ dependencies are found. This means that class dependencies are not checked.

Final thoughts

This is by no means a complete solution, but I hope it is of some use. Please feel free to comment, or suggest improvements.

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

SimpleCrypt PHP class for simple cryptography

I have just pushed a simple class called SimpleCrypt to GitHub:

https://github.com/glenscott/simple-crypt

This is an OO wrapper around the most common mcrypt functions. An example of use:

require_once 'SimpleCrypt.php';

// show the algorithms available for use
print_r( SimpleCrypt::get_available_algorithms() );

// encrypt a string using Triple DES (CBC mode)
$crypt = new SimpleCrypt( 'tripledes', 'mysecretkey' );
$encrypted_data = $crypt->encrypt_data( 'stringtoencrypt' );
echo "Encrypted data: " . bin2hex( $encrypted_data ) . "\n";
echo "Decrypted data: " . $crypt->decrypt_data( $encrypted_data ) . "\n";

If you find it useful, please let me know!

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Getting started with MongoDB and PHP on Mac OS X

NYみやげのMongo goods貰った!

Introduction

MongoDB is a document-orientated database written with scalability and high-performance in mind. It is one of a growing number of NoSQL systems – a database that does not rely on SQL or relational theory at all.

Getting a MongoDB server working with PHP on Mac OS X is relatively straightforward, and this tutorial shows you how.

Installing the MongoDB Server

The first step requires you to download and install the actual MongoDB system. The example shown below downloads v2.0.0 64 bit binaries for OS X. For other binaries, please check out the MongoDB Downloads page.

cd /tmp
curl -O http://fastdl.mongodb.org/osx/mongodb-osx-x86_64-2.0.0.tgz
tar zxvf mongodb-osx-x86_64-2.0.0.tgz 
sudo mv mongodb-osx-x86_64-2.0.0 /usr/local/mongodb
mkdir /usr/local/mongodb/data

Configuring the MongoDB server

We need to create a small configuration file so that MongoDB knows where its data files reside. Create the file /usr/local/mongodb/mongod.conf and add the following line:

dbpath = /usr/local/mongodb/data

Starting MongoDB

To manually start the MongoDB server, use the following command:

/usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongod.conf

This will start the server and will by default log all output to STDOUT.

Installing the MongoDB PHP Extension

Installing the PHP extension is simple:

sudo pecl install mongo

Once this has completed, add the following line to your /etc/php.ini file:

extension=mongo.so

Restart apache using sudo apachectl restart, and the extension should be available. This can be verified with the phpinfo call:

MongoDB extension for PHP

Example PHP script

To test your setup, the following simple script can be used to create a new collection and add two new records:

    <?php
    
    // connect
    $m = new Mongo();
    
    // select a database
    $db = $m->comedy;
    
    // select a collection (analogous to a relational database's table)
    $collection = $db->cartoons;
    
    // add a record
    $obj = array( "title" => "Calvin and Hobbes", "author" => "Bill Watterson" );
    $collection->insert($obj);
    
    // add another record, with a different "shape"
    $obj = array( "title" => "XKCD", "online" => true );
    $collection->insert($obj);
    
    // find everything in the collection
    $cursor = $collection->find();
    
    // iterate through the results
    foreach ($cursor as $obj) {
        echo $obj["title"] . "\n";
    }
    
    ?>

The script should output the following:

Calvin and Hobbes
XKCD

And that’s it! You now have a working MongoDB setup on Mac OS X.

For more information on the PHP extension for MongoDB, please see the following pages in the PHP Manual:

PHP: Mongo – Manual

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Mac OS X Lion PHP upgrade – php.ini and Suhosin

If you have upgraded from Snow Leopard to the new OS X Lion, you will notice PHP has also been upgraded – from 5.2 to 5.3.

A couple of points that I noticed post-install. Firstly, my existing /etc/php.ini file was moved to /etc/php.ini-5.2-previous. Restoring this was trivial:

sudo cp /etc/php.ini-5.2-previous /etc/php.ini

However, I noticed that extensions previously installed under /usr/lib/php/extensions/no-debug-non-zts-20090626/ had been removed. So, if you have extensions that you wish to still use with 5.3, they will need to be re-built.

I also noticed that Apple have included the Suhosin patch and extension. Suhosin is part of the Hardened-PHP project which aims to protect PHP applications against buffer-overflow and format string vulnerabilities. In theory, this functionality should be transparent to your application – no configuration or code changes are required.

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn

Install mcrypt PHP extension on OS X Snow Leopard

Lock away the evil...

Please note: The following instructions apply to Mac OS X 10.6 (Snow Leopard). I have an updated guide for how to install mcrypt on 10.7 (Lion).

mcrypt is a useful extension to PHP if you would like to support a wide range of encryption algorithms within your code.

This guide explains how you can enable install mcrypt, along with the PHP extension, on Mac OS X 10.6.

Xcode

The Xcode package installs the necessary versions of tools like autoconf which is needed during the PHP extension compilation process. Make sure you have at least Xcode v3.2 installed; the install package is available on the Snow Leopard install DVD under the “Optional Installs” folder.

libmcrypt

libmcrypt is the library that provides the encryption functions. We need to install this before building the PHP extension.

  • Open up Terminal.app
  • export CFLAGS="-arch x86_64"
  • Download libmcrypt from sourceforge
  • Unpack the archive somewhere
  • cd libmcrypt
  • ./configure --disable-shared
  • make
  • sudo make install

PHP Extension

Once we have libmcrypt installed, we can build the PHP extension.

You should see output similar to the following:

Configuring for:
PHP Api Version:         20090626
Zend Module Api No:      20090626
Zend Extension Api No:   220090626
  • ./configure
  • make
  • sudo make install

On successful install, you will see the following message:

Installing shared extensions:     /usr/lib/php/extensions/no-debug-non-zts-20090626/

To enable the module, include the following line in your /etc/php.ini file:

extension = mcrypt.so

Finally, restart your Apache server:

sudo apachectl restart

And that’s it: you should now be able to utilise the functions, as detailed in the PHP mcrypt documentation.

mcrypt php extension

Glen Scott

I’m a freelance software developer with 18 years’ professional experience in web development. I specialise in creating tailor-made, web-based systems that can help your business run like clockwork. I am the Managing Director of Yellow Square Development.

More Posts

Follow Me:
TwitterFacebookLinkedIn