Text

Using the Symfony Expression Language for a Reward Rules Engine

We recently adopted the Symfony Expression Language in the rules engine at OpenSky. It has brought a new level of flexibility to our system and creating new logic has never been easier.

Installing the expression language in your application is easy with composer. Just add the following to your composer.json:

"symfony/expression-language": "2.5.*@dev"

The expression language allows you to perform expressions that get evaluated with raw PHP code and return a single value. It can be any type of value and is not limited to boolean values. Here is a simple example:

use Symfony\Component\ExpressionLanguage\ExpressionLanguage;

$language = new ExpressionLanguage();

$expression = 'user["isActive"] == true and product["price"] > 20';
$context = array(
    'user' => array(
        'isActive' => true
    ),
    'product' => array(
        'price' => 30
    ),
);

$return = $language->evaluate($expression, $context);

var_export($return); // true

That is a very simple example on how to use the raw expression language. Now I will try to demonstrate how you could model a real implementation using Doctrine to persist your rules to a database, the Symfony Event Dispatcher to evaluate your rules and execute actions when your expressions evaluate to true.

To get started create a new Rule class and map it to a database using one of the Doctrine object mappers. For this example we will map it using the MongoDB ODM:

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;

/**
 * @ODM\Document
 */
class Rule
{
    /**
     * @ODM\Id
     */
    private $id;

    /**
     * @ODM\Collection
     */
    private $eventNames = array();

    /**
     * @ODM\String
     */
    private $expression;

    /**
     * @ODM\Collection
     */
    private $actionEvents = array();

    // ...
}

Now imagine you already have an event named user.add_to_cart being notified in your application. It looks something like this:

use Symfony\Component\EventDispatcher\Event;

class UserAddToCartEvent extends Event
{
    const onUserAddToCart = 'user.add_to_cart';

    private $user;
    private $product;

    // ...
}

class AddToCartController
{
    // ...

    public function addToCartAction($productId)
    {
        // ...

        $this->dispatcher->dispatch(UserAddToCartEvent::onUserAddToCart, new UserAddToCartEvent($user, $product));
    }
}

Say you want to give a reward to users who add items to their cart when they have loved more than 20 items and the price of the product is greater than 50 dollars. The Rule model we created earlier allows us to define a rule that will be executed when UserAddToCart::onUserAddToCart is dispatched:

$rule = new Rule();

// set the events this rule should be executed on.
$rule->setEventNames(array(
    UserAddToCartEvent::onUserAddToCart
));

// set the expression to evaluate when the rule is executed.
// if the user has loved more than 20 items and the price of the product is more than 50 dollars.
// the expression string will be evaluated by the Symfony expression language.
$rule->setExpression('event.getUser().getNumLoves() > 20 and event.getProduct().getPrice() > 50');

// set the action events to dispatch when the expression evaluates to true.
$rule->setActionEvents(array(
    array(
        'eventName' => UserCreditRewardEvent::onUserCreditReward,
        'recipientExpression' => 'event.getUser()',
        'attributes' => array(
            'amount' => 50
        ),
    )
));

$dm->persist($rule);
$dm->flush();

The above example assumes you have a UserCreditRewardEvent setup and a listener setup to process the event to give the user a credit. Here is what the event would look like:

use Symfony\Component\EventDispatcher\Event;

class UserCreditRewardEvent extends Event
{
    const onUserCreditReward = 'user.credit_reward';

    private $user;
    private $amount;

    // ...
}

And here is what the listener would look like to give the user the credit. This example assumes you already have a CreditManager service with an issueCredit() method you can use to give a user a credit for a dollar amount:

class UserCreditRewardListener
{
    private $creditManager;

    // ...

    public function onUserCreditReward(UserCreditRewardEvent $event)
    {
        $this->creditManager->issueCredit(
            $event->getUser(),
            $event->getAmount()
        );
    }
}

Now to bring it all together we need a RuleSubcriber to lookup the Rule objects from the database when events occur in our application. This class will evaluate the rules and then dispatch the resulting action events when the expressions return true.

use Symfony\Component\EventDispatcher\Event;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;

class RuleSubcriber implements EventSubscriberInterface
{
    private $dm;
    private $expressionLanguage;
    private $actionEventFactory;

    // ...

    public static function getSubscribedEvents()
    {
        return array(
            UserAddToCartEvent::onUserAddToCart => array('handleEvent', 0),
        );
    }

    public function handleEvent(Event $event)
    {
        $rules = $this->findRulesByEventName($event->getName());

        foreach ($rules as $rule) {
            if ($this->evaluateRule($rule, $event)) {
                $this->dispatchActionEvents($rule, $event);
            }
        }
    }

    private function findRulesByEventName($eventName)
    {
        return $this->dm->createQueryBuilder()
            ->field('eventNames')->equals($eventName)
            ->getQuery()
            ->execute();
    }

    private function evaluateRule(Rule $rule, Event $event)
    {
        return $this->expressionLanguage->evaluate($rule->getExpression(), array(
            'event' => $event,
        ));
    }

    private function dispatchActionEvents(Rule $rule, Event $event)
    {
        foreach ($rule->getActionEvents() as $action) {
            $this->dispatchActionEvent($action, $rule, $event);
        }
    }

    private function dispatchActionEvent(array $action, Rule $rule, Event $event)
    {
        $recipientUser = $this->expressionLanguage($action['recipientExpression'], array(
            'event' => $event,
        ));

        $actionEvent = $this->actionEventFactory->createActionEvent(
            $action,
            $recipientUser,
            $rule
        );

        $this->dispatcher->dispatch($action['eventName'], $actionEvent);
    }
}

The ActionEventFactory used in the above RuleSubcriber is a simple service used to create the action events we dispatch for our rules.

class ActionEventFactory
{
    public function createActionEvent(array $action, User $user, Rule $rule)
    {
        switch ($action['eventName']) {
            // ...

            case UserCreditRewardEvent::onUserCreditReward:
                return new UserCreditRewardEvent($user, $action['attributes']['amount']);
        }
    }
}

That is it! Now you have the ability to define rules that can be created with a user interface in your application and stored in a database. These rules get evaluated when certain events are dispatched within your application. When those rules evaluate to true you can dispatch other events that can give out credits, give free shipping, send e-mails, or do anything you can possibly imagine. Build up a repository of common actions as events and allow your business people to define new rules and rewards for promotional campaigns without having to involve a software engineer.

Text

Tailing Log Files Across Multiple Servers

Install the utility named dsh:

apt-get install dsh
brew install dsh

Add the list of servers you want to work with to .dsh/machines.list in your home directory. It might look something like this:

web1.prod.domain.com
web2.prod.domain.com
web3.prod.domain.com
web4.prod.domain.com
web5.prod.domain.com
web6.prod.domain.com

Add this to your .ssh/config:

Host web*.prod.domain.com
  User your_username

Now you can do things like this:

dsh -Mac -- "tail -f /var/log/some_log_file.log"

Combine that with grep to look for certain things that you are logging:

dsh -Mac -- "tail -f /var/log/some_log_file.log" | grep "look for something"
Text

Doctrine is not just an ORM for Relational Databases

In April of 2010 the first commit for the Doctrine MongoDB ODM project was made. I was experimenting with MongoDB at the time and I wanted to see how difficult it would be to build a version of Doctrine for MongoDB.

Up until the MongoDB ODM, Doctrine was solely a project built around the DBAL/ORM and was advertised as such. In May of 2010 we decided to widen the scope of the project so that we could host libraries like the MongoDB ODM. This change led to a spur of new contributors and development and we now have several object mappers developed under Doctrine and things are more active than ever.

Below is an overview of all the libraries underneath the Doctrine project.

Common Shared Libraries

doctrine/common

Doctrine Common contains some base functionality and interfaces you need in order to create a Doctrine style object mapper. All of our mapper projects follow the same Doctrine\Common\Persistence interfaces. Here are the ObjectManager and ObjectRepository interfaces:

<?php

namespace Doctrine\Common\Persistence

interface ObjectManager
{
    public function find($className, $id);
    public function persist($object);
    public function remove($object);
    public function merge($object);
    public function clear($objectName = null);
    public function detach($object);
    public function refresh($object);
    public function flush();
    public function getRepository($className);
}

interface ObjectRepository
{
    public function find($id);
    public function findAll();
    public function findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null);
    public function findOneBy(array $criteria);
}

doctrine/collections

Doctrine Collections is a library that contains classes for working with arrays of data. Here is an example using the simple Doctrine\Common\Collections\ArrayCollection class:

<?php

$data = new \Doctrine\Common\Collections\ArrayCollection(array(1, 2, 3));
$data = $data->filter(function($count) { return $count > 1; });

doctrine/annotations

Doctrine Annotations is a library that allows you to parse structured information out of a doc block.

Imagine you have a class with a doc block like the following:

<?php

/** @Foo(bar="value") */
class User
{

}

You can parse the information out of the doc block for User easily. Define a new annotation object:

<?php

/**
 * @Annotation
 * @Target("CLASS")
 */
class Foo
{
    /** @var string */
    public $bar;
}

Now you can get instances of Foo defined on the User:

<?php

$reflClass = new ReflectionClass('User');
$reader = new \Doctrine\Common\Annotations\AnnotationReader();
$classAnnotations = $reader->getClassAnnotations($reflClass);

foreach ($classAnnotations AS $annot) {
    if ($annot instanceof Foo) {
        echo $annot->bar; // prints "value";
    }
}

doctrine/inflector

Doctrine Inflector is a library that can perform string manipulations with regard to upper/lowercase and singular/plural forms of words.

<?php

$camelCase = 'camelCase';
$table = \Doctrine\Common\Inflector::tableize($camelCase);
echo $table; // camel_case

doctrine/lexer

Doctrine Lexer is a library that can be used in Top-Down, Recursive Descent Parsers. This lexer is used in Doctrine Annotations and in Doctrine ORM (DQL).

Here is what the AbstractLexer provided by Doctrine looks like:

<?php

namespace Doctrine\Common\Lexer;

abstract class AbstractLexer
{
    public function setInput($input);
    public function reset();
    public function resetPeek();
    public function resetPosition($position = 0);
    public function isNextToken($token);
    public function isNextTokenAny(array $tokens);
    public function moveNext();
    public function skipUntil($type);
    public function isA($value, $token);
    public function peek();
    public function glimpse();
    public function getLiteral($token);

    abstract protected function getCatchablePatterns();
    abstract protected function getNonCatchablePatterns();
    abstract protected function getType(&$value);
}

To implement a lexer just extend the Doctrine\Common\Lexer\AbstractParser class and implement the getCatchablePatterns, getNonCatchablePatterns, and getType methods. Here is a very simple example lexer implementation named CharacterTypeLexer. It tokenizes a string to T_UPPER, T_LOWER and T_NUMER:

<?php

use Doctrine\Common\Lexer\AbstractParser;

class CharacterTypeLexer extends AbstractLexer
{
    const T_UPPER =  1;
    const T_LOWER =  2;
    const T_NUMBER = 3;

    protected function getCatchablePatterns()
    {
        return array(
            '[a-bA-Z0-9]',
        );
    }

    protected function getNonCatchablePatterns()
    {
        return array();
    }

    protected function getType(&$value)
    {
        if (is_numeric($value)) {
            return self::T_NUMBER;
        }

        if (strtoupper($value) === $value) {
            return self::T_UPPER;
        }

        if (strtolower($value) === $value) {
            return self::T_LOWER;
        }
    }
}

Use CharacterTypeLexer to extract an array of upper case characters:

<?php

class UpperCaseCharacterExtracter
{
    private $lexer;

    public function __construct(CharacterTypeLexer $lexer)
    {
        $this->lexer = $lexer;
    }

    public function getUpperCaseCharacters($string)
    {
        $this->lexer->setInput($string);
        $this->lexer->moveNext();

        $upperCaseChars = array();
        while (true) {
            if (!$this->lexer->lookahead) {
                break;
            }

            $this->lexer->moveNext();

            if ($this->lexer->token['type'] === CharacterTypeLexer::T_UPPER) {
                $upperCaseChars[] = $this->lexer->token['value'];
            }
        }

        return $upperCaseChars;
    }
}

$upperCaseCharacterExtractor = new UpperCaseCharacterExtracter(new CharacterTypeLexer());
$upperCaseCharacters = $upperCaseCharacterExtractor->getUpperCaseCharacters('1aBcdEfgHiJ12');

print_r($upperCaseCharacters);

The variable $upperCaseCharacters contains all of the upper case characters:

Array
(
    [0] => B
    [1] => E
    [2] => H
    [3] => J
)

doctrine/cache

Doctrine Cache is a library that provides an interface for caching data. It comes with implementations for some of the most popular caching data stores. Here is what the Cache interface looks like:

<?php

namespace Doctrine\Common\Cache;

interface Cache
{
    function fetch($id);
    function contains($id);
    function save($id, $data, $lifeTime = 0);
    function delete($id);
    function getStats();
}

Here is an example using memcache:

<?php

$memcache = new \Memcache();
$cache = new \Doctrine\Common\Cache\MemcacheCache();
$cache->setMemcache($memcache);

$cache->set('key', 'value');

echo $cache->get('key') // prints "value"

Other supported drivers are:

  • APC
  • Couchbase
  • Filesystem
  • Memcached
  • MongoDB
  • PhpFile
  • Redis
  • Riak
  • WinCache
  • Xcache
  • ZendData

Database Abstraction Layers

doctrine/dbal

Doctrine DBAL is a library that provides an abstraction layer for relational databases in PHP. Read Doctrine DBAL: PHP Database Abstraction Layer blog post for more information on the DBAL.

doctrine/mongodb

Doctrine MongoDB is a library that provides an abstraction layer on top of the PHP MongoDB PECL extension. It provides some additional functionality and abstractions to make working with MongoDB easier.

doctrine/couchdb-client

Doctrine CouchDB Client is a library that provides a connection abstraction to CouchDB by wrapping around the CouchDB HTTP API.

<?php

$client = \Doctrine\CouchDB\CouchDBClient::create();

array($id, $rev) = $client->postDocument(array('foo' => 'bar'));
$client->putDocument(array('foo' => 'baz'), $id, $rev);

$doc = $client->findDocument($id);

Object Mappers

The object mappers are where all the pieces come together. The object mappers provide transparent persistence for PHP objects. As mentioned above, they all implement the common interfaces from Doctrine\Common so working with each of them is generally the same. You have an ObjectManager to manage the persistent state of your domain objects:

<?php

$user = new User();
$user->setId(1);
$user->setUsername('jwage');

$om = $this->getYourObjectManager();
$om->persist($user);
$om->flush(); // insert the new document

Then you can find that object later and modify it:

<?php

$user = $om->find('User', 1);
echo $user->getUsername(); // prints "jwage"

$user->setUsername('jonwage'); // change the obj in memory

$om->flush(); // updates the object in the database

You can find more information about the supported object mappers below:

doctrine/orm

Doctrine ORM provides persistence for PHP objects to relational database.

doctrine/couchdb-odm

Doctrine CouchDB ODM provides persistence for PHP objects to CouchDB.

doctrine/phpcr-odm

Doctrine PHPCR (Content Repository) ODM provides persistence to a backend like Jackalope or Midgard2. This is a specialized object mapper for dealing with data designed for building content websites. Think of this like a backend for a CMS (Content Management System).

doctrine/mongodb-odm

Doctrine MongoDB ODM provides persistence for PHP objects to MongoDB. You can read more about the MongoDB ODM here.

doctrine/orientdb-odm

Doctrine OrientDB ODM provides persistence for PHP objects to OrientDB.

Text

Sending Safari Push Notifications with PHP

When OSX Mavericks was released, I was excited to see that push notifications were supported within Safari. So now, just like iOS, we can send push notifications to Safari regardless of whether the browser is open or not. This post gives overview of how to get them working from PHP using the PHP APNS library.

Registering with Apple and Generating Certificates

This post assumes you have read this article from Apple and have already generated your security certificates.

PHP APNS Library

The PHP APNS (Apple Push Notification Service) library makes it extremely easy to get Safari push notifications working on your website. Just install it with composer to get started:

Installation

composer require jwage/php-apns
composer install

Push Package

Once you have the package installed you can start integrating it with your website. The first thing we need to do is create a base push package for your website. From Apple’s website:

When a user is asked for permission to receive push notifications, Safari asks your web server for a package. The package contains data that is used by the notification UI, such as your website name and icon, as well as a cryptographic signature. The signature verifies that your notification hasn’t been intercepted by a man-in-the-middle attack and that it is indeed coming from a trusted source: you.

You can find a sample base push package within the PHP APNS library here. Copy this somewhere in your application and customize the icons in the icon.iconset folder and the website.json:

{
    "websiteName": "WebsiteName",
    "websitePushID": "web.com.domain",
    "allowedDomains": ["http://{{ host }}", "https://{{ host }}"],
    "urlFormatString": "http://{{ host }}/%@",
    "authenticationToken": "{{ userId }}",
    "webServiceURL": "https://{{ host }}/safari_push_notifications/{{ userId }}"
}

Web Service Endpoints

Now it is time to setup some endpoints in your web application for Safari to communicate with. You need endpoints to do the following:

  • Generate a push package for an individual user.
  • Register a users device token.
  • Deregister a users device token.
  • Record log data sent by Safari when errors occur.

Here is a pseudo controller demonstrating what each endpoint needs to do:

<?php

namespace App\Controller;

use JWage\APNS\Certificate;
use JWage\APNS\Safari\PackageGenerator;

class SafariPushNotificationsController
{
    public function packageAction($userId)
    {
        // Send push notification package to browser when Safari asks user for permission to send you notifications.

        $certificate = new Certificate(file_get_contents('apns.p12'), 'certpassword');
        $packageGenerator = new PackageGenerator(
            $certificate, '/path/to/base/pushPackage/path', 'yourhost.com'
        );

        // returns JWage\APNS\Safari\Package instance
        $package = $packageGenerator->createPushPackageForUser('userid');

        // send $package->getZipPath() to the browser
    }

    public function registerAction($userId, $deviceToken)
    {
        // store $deviceToken on the $userId so you can use it later to send pushes
    }

    public function deregisterAction($userId, $deviceToken)
    {
        // remove $deviceToken from $userId
    }

    public function logAction($userId)
    {
        // log information sent for debugging purposes
    }
}

Requesting Permission

Requesting permission to send a user notifications in Safari can be done using this little snippet of javascript on your website:

if ('safari' in window && 'pushNotification' in window.safari) {
    var checkRemotePermission = function (permissionData) {
        if (permissionData.permission === 'default') {
            window.safari.pushNotification.requestPermission(
                'http://domain.com/safari_push_notifications/{{ userId }}',
                'web.com.domain',
                {
                    'userId': {{ userId }}
                },
                checkRemotePermission
            );
        } else if (permissionData.permission === 'denied') {
            // do something when permission is denied
        } else if (permissionData.permission === 'granted') {
            // do something when permission is granted
        }
    };

    // Ensure that the user can receive Safari Push Notifications.
    var permissionData = window.safari.pushNotification.permission('web.com.domain');
    checkRemotePermission(permissionData);
}

When a user visits your website with Safari in OSX Mavericks it will hit your web services endpoint, download the push package and ask the user if they want to receive notifications from your website.

Sending Push Notifications

Once you have done all of the above, you should be ready to send push notifications. The PHP APNS library makes this extremely easy. Here is an example:

use JWage\APNS\Certificate;
use JWage\APNS\Client;
use JWage\APNS\Sender;
use JWage\APNS\SocketClient;

$certificate = new Certificate(file_get_contents('apns.pem'));
$socketClient = new SocketClient($certificate, 'gateway.push.apple.com', 2195);
$client = new Client($socketClient);
$sender = new Sender($client);

$sender->send('devicetoken', 'Title of push', 'Body of push', 'http://deeplink.com');

You can create an easy to use service in your application for sending push notifications to an instance of User, assuming it implements a getSafariDeviceToken() method:

class SafariPushNotificationSender
{
    private $sender;

    public function __construct(Sender $sender)
    {
        $this->sender = $sender;
    }

    public function sendToUser(User $user, $title, $body, $link)
    {
        return $this->sender->send($user->getSafariDeviceToken(), $title, $body, $link);
    }
}

Now it is as simple as the following:

$safariPushNotificationSender = new SafariPushNotificationSender($sender);
$safariPushNotificationSender->sendToUser($user, 'Title of push', 'Body of push', 'http://deeplink.com');

I hope this was helpful! If you have any questions please leave them in the comments. Enjoy!

Text

MongoDB PHP MongoDate Tricks

Here are a few tricks I’ve learned working with MongoDB and PHP.

Create DateTime from MongoId

Because MongoDB identifiers contain the date you can easily create a DateTime instance from them.

public function getDateTimeFromMongoId(MongoId $mongoId)
{
    $dateTime = new DateTime('@'.$mongoId->getTimestamp());
    $dateTime->setTimezone(new DateTimeZone(date_default_timezone_get()));
    return $dateTime;
}

This is useful when you don’t want to create an additional field to store the date a document was created. You can also use the _id index that already exists to paginate and filter/sort by date.

Create MongoId with Date in the Past

For the same reason as above, you can create a MongoId instance with a date in the past. This was copied from a stackoverflow answer by Derek Rethans.

public createMongoIdFromTimestamp($timestamp)
{
    $ts = pack('N', $timestamp);
    $m = substr(md5(gethostname()), 0, 3);
    $pid = pack('n', posix_getpid());
    $trail = substr(pack('N', $this->inc++), 1, 3);

    $bin = sprintf('%s%s%s%s', $ts, $m, $pid, $trail);

    $id = '';
    for ($i = 0; $i < 12; $i++ ) {
        $id .= sprintf('%02X', ord($bin[$i]));
    }

    return new \MongoID($id);
}

This was useful when migrating legacy data in to a collection where we utilize the _id for pagination and displaying a created date. If we simply created identifiers with todays date then it would show old records with all the same date and pagination/sorting would be broken.

Text

Quotes from The Road to Serfdom by Friedrich Hayek

I recently finished reading The Road to Serfdom by Friedrich Hayek. Here are some parts that stood out to me.

"We are ready to accept almost any explanation of the present crisis of our civilization except one: that the present state of the world may be the result of genuine error on our own part and that the pursuit of some of our most cherished ideals has apparently produced results utterly different from those which we expected."

“Democracy extends the sphere of individual freedom,” he said in 1848; “socialism restricts it. Democracy attaches all possible value to each man; socialism makes each man a mere agent, a mere number. Democracy and socialism have nothing in common but one word: equality. But notice the difference: while democracy seeks equality in liberty, socialism seeks equality in restraint and servitude.”

"While it is true, of course, that inventions have given us tremendous power, it is absurd to suggest that we must use this power to destroy our most precious inheritance: liberty. It does mean, however, that if we want to preserve it, we must guard it more jealously than ever and that we must be prepared to make sacrifices for it."

"The “social goal,” or “common purpose,” for which society is to be organized is usually vaguely described as the “common good,” the “general welfare,” or the “general interest.” It does not need much reflection to see that these terms have no sufficiently definite meaning to determine a particular course of action. The welfare and the happiness of millions cannot be measured on a single scale of less and more. The welfare of a people, like the happiness of a man, depends on a great many things that can be provided in an infinite variety of combinations. It cannot be adequately expressed as a single end, but only as a hierarchy of ends, a comprehensive scale of values in which every need of every person is given its place."

"Although each individual might wish the state to act in some way, there will be almost as many views about what the government should do as there are different people."

"It is not difficult to see what must be the consequences when democracy embarks upon a course of planning which in its execution requires more agreement than in fact exists. The people may have agreed on adopting a system of directed economy because they have been convinced that it will produce great prosperity."

"But the democratic legislature will long hesitate to relinquish the decisions on really vital issues, and so long as it does so it makes it impossible for anyone else to provide the comprehensive plan. Yet agreement that planning is necessary, together with the inability of democratic assemblies to produce a plan, will evoke stronger and stronger demands that the government or some single individual should be given powers to act on their own responsibility."

"The belief is becoming more and more widespread that, if things are to get done, the responsible authorities must be freed from the fetters of democratic procedure."

"Hitler did not have to destroy democracy; he merely took advantage of the decay of democracy and at the critical moment obtained the support of many to whom, though they detested Hitler, he yet seemed the only man strong enough to get things done."

"Nothing distinguishes more clearly conditions in a free country from those in a country under arbitrary government than the observance in the former of the great principles known as the Rule of Law. Stripped of all technicalities, this means that government in all its actions is bound by rules fixed and announced beforehand—rules which make it possible to foresee with fair certainty how the authority will use its coercive powers in given circumstances and to plan one’s individual affairs on the basis of this knowledge."

"Hence the familiar fact that the more the state “plans,” the more difficult planning becomes for the individual."

"Those most immediately interested in a particular issue are not necessarily the best judges of the interests of society as a whole."

"The important thing is that the rule enables us to predict other people’s behavior correctly, and this requires that it should apply to all cases—even if in a particular instance we feel it to be unjust."

"Man is free if he needs to obey no person but solely the laws."

"The idea that there is no limit to the powers of the legislator is in part a result of popular sovereignty and democratic government. It has been strengthened by the belief that, so long as all actions of the state are duly authorized by legislation, the Rule of Law will be preserved. But this is completely to misconceive the meaning of the Rule of Law. This rule has little to do with the question whether all actions of government are legal in the juridical sense. They may well be and yet not conform to the Rule of Law."

"If the law says that such a board or authority may do what it pleases, anything that board or authority does is legal—but its actions are certainly not subject to the Rule of Law. By giving the government unlimited powers, the most arbitrary rule can be made legal; and in this way a democracy may set up the most complete despotism imaginable."

"Indeed, when security is understood in too absolute a sense, the general striving for it, far from increasing the chances of freedom, becomes the gravest threat to it."

"There is no reason why in a society which has reached the general level of wealth which ours has attained the first kind of security should not be guaranteed to all without endangering general freedom."

"We must here return for a moment to the position which precedes the suppression of democratic institutions and the creation of a totalitarian regime. In this stage it is the general demand for quick and determined government action that is the dominating element in the situation, dissatisfaction with the slow and cumbersome course of democratic procedure which makes action for action’s sake the goal. It is then the man or the party who seems strong and resolute enough “to get things done” who exercises the greatest appeal. “Strong” in this sense means not merely a numerical majority—it is the ineffectiveness of parliamentary majorities with which people are dissatisfied. What they will seek is somebody with such solid support as to inspire confidence that he can carry out whatever he wants. It is here that the new type of party, organized on military lines, comes in."

"That socialism can be put into practice only by methods which most socialists disapprove is, of course, a lesson learned by many social reformers in the past."

"It seems to be almost a law of human nature that it is easier for people to agree on a negative program—on the hatred of an enemy, on the envy of those better off— than on any positive task."

"The definitely antagonistic attitude which most planners take toward internationalism is further explained by the fact that in the existing world all outside contacts of a group are obstacles to their effectively planning the sphere in which they can attempt it. It is therefore no accident that, as the editor of one of the most comprehensive collective studies on planning has discovered to his chagrin, “most ‘planners’ are militant nationalists.”

"The desire to organize social life according to a unitary plan itself springs largely from a desire for power."

"We have seen before how the separation of economic and political aims is an essential guaranty of individual freedom and how it is consequently attacked by all collectivists. To this we must now add that the “substitution of political for economic power” now so often demanded means necessarily the substitution of power from which there is no escape for a power which is always limited. What is called economic power, while it can be an instrument of coercion, is, in the hands of private individuals, never exclusive or complete power, never power over the whole life of a person. But centralized as an instrument of political power it creates a degree of dependence scarcely distinguishable from slavery."

"It would, however, be highly unjust to regard the masses of the totalitarian people as devoid of moral fervor because they give unstinted support to a system which to us seems a denial of most moral values. For the great majority of them the opposite is probably true: the intensity of the moral emotions behind a movement like that of National Socialism or communism can probably be compared only to those of the great religious movements of history."

"Yet while there is little that is likely to induce men who are good by our standards to aspire to leading positions in the totalitarian machine, and much to deter them, there will be special opportunities for the ruthless and unscrupulous. There will be jobs to be done about the badness of which taken by themselves nobody has any doubt, but which have to be done in the service of some higher end, and which have to be executed with the same expertness and efficiency as any others. And as there will be need for actions which are bad in themselves, and which all those still influenced by traditional morals will be reluctant to perform, the readiness to do bad things becomes a path to promotion and power. The positions in a totalitarian society in which it is necessary to practice cruelty and intimidation, deliberate deception and spying, are numerous. Neither the Gestapo nor the administration of a concentration camp, neither the Ministry of Propaganda nor the SA or SS (or their Italian or Russian counterparts), are suitable places for the exercise of humanitarian feelings.13 Yet it is through positions like these that the road to the highest positions in the totalitarian state leads. It is only too true when a distinguished American economist concludes from a similar brief enumeration of the duties of the authorities of a collectivist state that “they would have to do these things whether they wanted to or not: and the probability of the people in power being individuals who would dislike the possession and exercise of power is on a level with the probability that an extremely tender-hearted person would get the job of whipping-master in a slave plantation.”

Text

Tracking New Member Origination with Symfony2

NOTE The example code in this blog post has been simplified to make things more concise and easy to read.

At OpenSky it is important for us to track how a member joined so that we can determine causation between joining and any subsequent actions taken on the site, like an order.

To get started create a new class called OriginationManager with a method named updateHistory. It will accept a Request object and record the query parameters from the current URL in the session.

<?php

class OriginationManager
{
    public function updateHistory(Request $request)
    {
        $session = $request->getSession();

        $history = $session->get('user_origination_history', array());
        $history[] = $request->query->all();

        $session->set('user_origination_history', $history);
    }
}

Now create a OriginationListener class that will listen to the kernel.request event and make use of the OriginationManager::updateHistory() API we just created.

<?php

class OriginationListener
{
    // ...

    public function onKernelRequest(GetResponseEvent $event)
    {
        $request = $event->getRequest();

        // Only process non-logged-in users
        if ($token = $this->securityContext->getToken()) {
            $user = $token->getUser();
            if ($user instanceof User) {
                return;
            }
        }

        $this->originationManager->updateHistory($request);
    }
}

Now we are tracking the query parameters of logged out users on every request. We can make use of this information later when a user joins. First, lets define a new model that can be used to store the origination information along with the user (assume the model is persisted with Doctrine). Each new User gets an Originator record when they join. It allows us to essentially see how a new user entered the site and what clicks they made before joining.

<?php

class Originator
{
    /**
     * The raw array of request query data.
     *
     * @var array
     */
    protected $history = array();

    /**
     * The user that originated this member.
     *
     * @var User
     */
    protected $user;

    /**
     * The value of the first osky_origin parameter we encounter.
     *
     * @var string
     */
    protected $origin;

    /**
     * The value of the first osky_source parameter we encounter.
     *
     * @var string
     */
    protected $source;

    // ...
}

Assume your application already notifies an event named user.created. In OriginationListener create an onUserCreated method that will listen to the user.created event.

<?php

class OriginationListener
{
    // ...

    public function onUserCreated(UserCreatedEvent $event)
    {
        $user = $event->getUser();
        $request = $event->getRequest();

        $this->originationManager->assignUserOrigination($user, $request);
    }
}

Next, create the OriginationManager::assignUserOrigination() method. It will utilize the request query parameters we saved on the session earlier to create a new Originator record.

<?php

class OriginationManager
{
    public function assignUserOrigination(User $user, Request $request)
    {
        $session = $request->getSession();

        $history = $session->get('user_origination_history', array());

        $originator = new Originator();
        $originator->setHistory($history);

        foreach ($history as $query) {
            if (!$originator->getOrigin() && isset($query['osky_origin'])) {
                $originator->setOrigin($query['osky_origin']);

                if ($user = $this->userRepository->find($query['osky_origin'])) {
                    $originator->setUser($user);
                }
            }

            if (!$originator->getSource() && isset($query['osky_source'])) {
                $originator->setSource($query['osky_source']);
            }
        }

        $user->setOriginator($originator);

        $session->remove('user_origination_history');
    }
}

Now if a logged out user were to visit OpenSky with a parameter named osky_origin in the URL with another users id as the value, the Originator record that gets created for the new member will have a reference to that User. We can then utilize that information do whatever we want. In our case we give the user that got the new User to join a credit and a thank you. The osky_source parameter can be used as an arbitrary reporting variable to help with identifying marketing campaigns.

Keep in mind that this is an example implementation and it omits many details specific to OpenSky for the sake of simplicity. The real implementation we use has dozens of other parameters that can be used to associate records together and assist with reporting. You can add to this implementation and check for custom parameters and standard ones like utm_source, utm_campaign, etc.

Text

Building Activity Streams using Symfony2, Doctrine2 and MongoDB

NOTE The example code in this blog post has been simplified to make things more concise and easy to read.

At OpenSky we utilize Symfony2, Doctrine2 and MongoDB for streams of activity across our site. We’ve based our implementation on the JSON Activity Streams 1.0 specification. This blog post demonstrates how you can easily setup your own activity streams.

An activity item consists of a published date, actor, object, target and verb. It describes an action performed by an entity. Here is an example JSON document of what an activity item might look like:

{
  "published": "2011-02-10T15:04:55Z",
  "actor": {
    "url": "http://example.org/martin",
    "objectType" : "person",
    "id": "tag:example.org,2011:martin",
    "image": {
      "url": "http://example.org/martin/image",
      "width": 250,
      "height": 250
    },
    "displayName": "Martin Smith"
  },
  "verb": "post",
  "object" : {
    "url": "http://example.org/blog/2011/02/entry",
    "id": "tag:example.org,2011:abc123/xyz"
  },
  "target" : {
    "url": "http://example.org/blog/",
    "objectType": "blog",
    "id": "tag:example.org,2011:abc123",
    "displayName": "Martin's Blog"
  }
}

To get started implementing this let’s define a base PHP class that we can use to represent an activity item.

<?php

namespace Project\Document;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;

/**
 * @ODM\MappedSuperclass(collection="activity_items")
 * @ODM\InheritanceType("SINGLE_COLLECTION")
 * @ODM\DiscriminatorField(fieldName="verb")
 * @ODM\DiscriminatorMap({
 *     "love" = "Project\Document\LoveActivityItem"
 * })
 */
abstract class AbstractActivityItem
{
    /** @ODM\Date */
    protected $published;

    protected $actor;
    protected $object;
    protected $target;

    // ...
}

For this blog post we’re only going to implement one verb called love. On OpenSky you can love products and the action is pushed in to the feed of your followers. So let’s get started implementing the LoveActivityItem class:

<?php

namespace Project\Document;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;
use Project\Document\Love;
use Project\Document\Product;
use Project\Document\User;

/** @ODM\Document */
class LoveActivityItem extends AbstractActivityItem
{
    /** @ODM\ReferenceOne(targetDocument="User") */
    protected $actor;

    /** @ODM\ReferenceOne(targetDocument="Love") */
    protected $object;

    /** @ODM\ReferenceOne(targetDocument="Product") */
    protected $target;

    public function __construct(
        User $user, Love $love, Product $product
    )
    {
        $this->actor = $user;
        $this->object = $love;
        $this->target = $product;
    }

    // ...
}

Now that we have our basic model defined, let’s implement the code that will wire everything up. Assume we already have a Symfony event in our application being notified called user.love. So setup a listener that listens to that event and records the activity item:

<?php

namespace Project/Listener;

class ActivityListener
{
    /**
     * @var Project\Activity\ActivityManager
     */
    private $activityManager;

    /**
     * Listens to the `user.love` event.
     */
    public function onUserLove(UserLoveEvent $event)
    {
        $this->activityManager->recordUserLove(
            $event->getUser(),
            $event->getLove()
        );
    }
}

Now let’s implement the ActivityManager class with the recordUserLove method:

<?php

namespace Project\Activity;

use Doctrine\ODM\MongoDB\DocumentManager;
use Project\Document\LoveActivityItem;
use Project\Document\Love;
use Project\Document\User;

class ActivityManager
{
    private $dm;

    public function __construct(DocumentManager $dm)
    {
        $this->dm = $dm;
    }

    public function recordUserLove(User $user, Love $love)
    {
        $this->dm->persist(
            new LoveActivityItem($user, $love, $love->getProduct())
        );
    }
}

The new activity item will get flushed to the database at the end of the action with the love since we flush in the controller where the user.love event is notified. Now it is possible to generate a feed of activity items for the whole site, a specific user or even a set of users:

<?php

$user = $dm->getRepository('Project\Document\User')->findOneByEmail('jonwage@gmail.com');

$activityItems = $dm->getRepository('Project\Document\ActivityItem')
    ->createQueryBuilder()
    ->field('actor')->references($user)
    ->getQuery()
    ->execute();

That is it. You can easily add new verbs to the model and add code to the listener and manager to record new activities. Good luck! I hope this was helpful for someone!

Text

SunshinePHP Miami

This past weekend I attended SunshinePHP in beautiful Miami, Florida. I was lucky enough to get to speak about our experiences building OpenSky with Symfony. The weather was perfect and I got to learn some new things as well. Below you can find the slides from my presentation.

Text

Doctrine Common Library

Doctrine started as a library where all the internal components were coupled together. But as things have evolved the components have been decoupled and shared between the projects. This change also makes it possible for other people to use these pieces of Doctrine even if they don’t use the ORM or any other project.

The Doctrine\Common namespace contains a few things like:

DocBlock AnnotationsLibrary

With the annotations library you can parse information out of your DocBlocks in to PHP objects. The object mapper projects use this feature for specifying entity mapping information in the DocBlocks of your classes, properties and methods. Here is an example of what an entity looks like in the ORM:

namespace MyProject\Entities;

use Doctrine\ORM\Mapping AS ORM;
use Symfony\Component\Validation\Constraints AS Assert;

/**
 * @ORM\Entity
 */
class User
{
    /**
     * @ORM\Id @ORM\Column @ORM\GeneratedValue
     */
    private $id;

    /**
     * @ORM\Column(type="string")
     * @Assert\NotEmpty
     * @Assert\Email
     */
    private $email;
}

Cache Drivers

The cache drivers provide a common interface to cache backends in PHP. Here are the supported drivers:

  • ApcCache
  • ArrayCache
  • FileCache
  • FilesystemCache
  • MemcacheCache
  • MemcachedCache
  • PhpFileCache
  • RedisCache
  • WinCacheCache
  • XcacheCache
  • ZendDataCache

The interface is very simple:

function fetch($id);
function contains($id);
function save($id, $data, $lifeTime = 0);
function delete($id);
function getStats();

Persistence Library

The persistence interfaces are implemented by the object mapper libraries. They provide the common base classes and interfaces that a Doctrine object persistence library should implement, such as:

ObjectManager

function find($className, $id);
function persist($object);
function remove($object);
function merge($object);
function clear($objectName = null);
function detach($object);
function refresh($object);
function flush();
function getRepository($className);
function getClassMetadata($className);
function getMetadataFactory();
function initializeObject($obj);
function contains($object);

ObjectRepository

function find($id);
function findAll();
function findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null);
function findOneBy(array $criteria);
function getClassName();

ClassMetadataFactory

function getAllMetadata();
function getMetadataFor($className);
function hasMetadataFor($className);
function setMetadataFor($className, $class);
function isTransient($className);

ClassMetadata

function getName();
function getIdentifier();
function getReflectionClass();
function isIdentifier($fieldName);
function hasField($fieldName);
function hasAssociation($fieldName);
function isSingleValuedAssociation($fieldName);
function isCollectionValuedAssociation($fieldName);
function getFieldNames();
function getIdentifierFieldNames();
function getAssociationNames();
function getTypeOfField($fieldName);
function getAssociationTargetClass($assocName);
function isAssociationInverseSide($assocName);
function getAssociationMappedByTargetField($assocName);
function getIdentifierValues($object);

The Doctrine\Common namespace contains lots more than I have mentioned here. So if you want to learn more check it out on GitHub or read the documentation. It is not that complete yet but it has some useful information.