File uploads not working with PhantomJS

Just a quick one today: when you struggle with writing Acceptance Tests for file uploads using PhantomJS (will version 2.5 ever come out? *sigh*), here are two possible solutions for you.

File inputs are ugly, let’s just not show them

The first issue might be that the input field is hidden for obvious reasons. Luckily we can solve this issue without making your designer cry (and we don’t want to do this, don’t we?) by just tweaking the CSS using JavaScript.

In my case, the input field was hidden by using a very low opacity value, so I set it to 1 just before attaching the actual file:

// PhantomJS 2.1 workaround #1:
// Make input[type=file] visible
$I->executeJS(
   'document.getElementById(\'input-field\').' . 
   'setAttribute(\'style\',\'opacity:1\');'
);
$I->attachFile('#input-field', 'test.jpg');

If this solves your problem – great! If not, I got one more.

Don’t rush – one at a time

While the first issue is somewhat understandable … well, maybe …, anyway, I found the second one just by good old trial-and-error: PhantomJS doesn’t like multiple uploads!

Now we don’t want to remove the possibility to upload multiple files just to please PhantomJS, so let’s do another tweak:

// Phantomjs 2.1 workaround #2:
// Don't use multiple uploads
$I->executeJS(
   'document.getElementById(\'input-field\').' . 
   'removeAttribute(\'multiple\');'
);

Of course we can’t test multiple uploads that way, which is a bit sad, but right now I’m afraid we have to live with that.

Did it blend?

If you found these two little tips helpful, or if you have another nasty PhantomJS workaround at hand, please drop a comment below!

Automated bad practice

One new feature of Laravel 5.4 are so called “Automatic Facades”. My colleague was expecting my reaction when he send me the link to the blog entry announcing this feature (I’m not linking it intentionally) with a thievish smile.

As Laravel’s documentation puts it, “Facades provide a static interface to classes that are available in the application’s service container”. In other words, you can access any class almost everywhere without hassling about nasty things like dependency injection. Hm? Do I hear Singleton somewhere?

So far it was required to add Facade classes in order to make this work. It seems even this little barrier has fallen now.

Laravel’s Facade in my opinion is one of the worst features a PHP framework offers nowadays. After times of Zend Framework 1 and the infamous Registry (I abused it too), when IoC finally arrived in the PHP world, it somehow seemed to me as quite a setback when Laravel became more and more popular with its Facade feature (it’s not even the Facade design pattern anyway).

Here’s why.

1. It’s tempting

Making classes available in the code everywhere is tempting unexperienced (and experienced) developers to use the facade everywhere, without thinking if this might be a good idea or not. I’ve seen things! I even found Facade in configuration files. IN CONFIG FILES WTF!!1

If you don’t pay close attention, your classes soon will violate the single responsibility principle, simply because you don’t really notice how many dependencies you suddenly use.

2. It binds your code

In my opinion code should be bound as less to the underlying framework as possible. I totally get the advantages of ORMs and such, I’m not opting against packages etc., but binding your code too close to a framework can cause problems if you want to use your business logic over longer terms.

But well, I guess if you are willing to use non-namespaced helper functions everywhere in your code (another Laravel feature I “deeply adore”), you don’t really care about this either.

3. It requires more overhead to test

“But Facades are more testable than traditional static methods!” I hear the Artisans yell. Yeah, sure, that is true.

What a benefit!

It just requires bootstrapping of Laravel for every test, making unit tests significantly slower. Your tests will become more complex and thus require more maintenance, as you will need to mock more dependencies which are all required to make one test work.

4. It’s actually not that convenient

Any modern IDE provides Code completion, so I can see in a second which methods a class offers. With Facade, I need an additional package that parses the Facades and generates documentation files that the IDE can parse for the Code completion.

How convenient!

When stepping through the code during debugging it annoys me pretty much to go through the facade over and over again. It’s just so unnecessary…

Probably both IDE and Debuggers are not so commonly used amongst Artisans?

5. It’s not more expressive

I really don’t get why

Cache::get('key')

is more “expressive” than

$this->cache->get('key')

Somebody needs to explain!

Fair enough

To be fair, even the official Laravel documentation states that Facade should not be used when building a third-party package, and that it bears some of the above mentioned risks. So Taylor is aware of it. But then why is this feature enforced in the first place anyway? I think it’s because Laravel claims to be so “expressive” and “beautiful” in it’s syntax that everybody accepts the downsides.

Decide for yourself if it is worth it.

Probably I’m just missing the point here? Why do you think Laravel’s Facade should be used rather than Dependency Injection?

DI container and IDE integration

Say we use a DI container directly, i.e. via functions like get() or make(), how do we achieve proper IDE support? For example I create a class instance the usual way

$server = $this->container->get(Server::class);

and I want to know what methods my Server class provides

$server-> ???

I could use docblocks

/** @var Server $server */ 
$server = $this->container->get(Server::class);

This works fine in many IDE like PhpStorm:

php-di-ide-integration-1

however the docblock is somehow annoying to me. The IDE *should* really know what class instance the variable $server is, since everything it needs to know is in the class name constant I provided as parameter for the get() function.

There might be a way using the hardly known .phpstorm.meta.php file for PhpStorm, but that one is currently some sort of under construction – at least I couldn’t get it working properly. Hopefully the PhpStorm guys will extend the documentation soon!

Once again, the PHP community provides the solution: the small but pretty cool PhpStorm PHP-DI plugin does exactly what we need! Now PhpStorm knows that $server is an instance of the Server class without me needing to do any further work:

php-di-ide-integration-2

Class name constants should be used wherever possible nowadays, however it also works with the Fully-qualified class name as string:

$server = $this
             ->container
             ->get('CarstenWindler\ApiGuy\Server');

But really, you shouldn’t do that. Please. Better just forget what I said. Forget it. Begone!

Awesome! Although actually written for the PHP-DI container, it also supports container-interop and even that crapp… I mean lovely but omnipresent Laravel App Facade sort of jun… container… thing.

I’d recommend to use DI containers directly only in very few places like Application bootstrapping or factories. But imagine you have to deal with a legacy Laravel 4.2 application which uses that awful App::make() basically EVERYWHERE, you are happy about any help.

Debug Unit Tests using the PhpStorm built-in test runner

PhpStorm comes with a handy phpunit test runner, which lets you execute your tests with just one click. Even better: if a test fails, you can just run that test also with just one click. That’s how I like it.

However I often use the debugger to debug failed tests. This is easy with phpunit and via the command line (see e.g. Remote CLI debugging). But since PhpStorm is obviously not using the command line, we have to pass the information necessary for Xdebug and PhpStorm to initiate the debug session using the Test Runners configuration.

XDEBUG_CONFIG=idekey=PHPSTORM ; PHP_IDE_CONFIG=serverName=local.dev

Needless to mention that the above values need to fit your setup. Most likely your server name will be different, and make sure to check the ide key.

But where to put it? Assuming you already setup your Run/Debug configuration, just open the configuration dialog and put the above environment variables into the “Environment variables” input field. You don’t need to add “export” or anything. Well, just have a look:

phpstorm_unit_test_runner

Now just place your breakpoints as you need them and start the Run/Debug configuration.

Debug your Codeception tests with Xdebug and PhpStorm

I use Xdebug not only to hunt bugs, but also for every day development work. Of course I also want to use it when I write my Tests.

I one of my current projects we use Codeception for both Unit- and Acceptance tests. Debugging Codeception Unit tests is as easy as for phpUnit, you just have to make sure that the XDEBUG_SESSION is set as described in my post about CLI debugging with Xdebug.

What to test

The Acceptance tests I want to debug are basically just testing a REST API, so it’s usually firing HTTP GET/POST requests against various endpoints:

class ApiCest
{
    public function testDoStuff(ApiTester $I)
    {
        $I->sendPOST('/my/api/dostuff');

        // testing starts here
        $I->seeResponseCodeIs(200);
    }
}

In order to debug the actual “dostuff” endpoint, I have to take care of two things: the debug session Cookie and the PhpStorm debug configuration.

Set the XDEBUG_SESSION cookie

The usual way for me to start a PhpStorm session for non-CLI scripts is to enable the “Start Listening for PHP Debug Connections” in PhpStorm and initiate a request with the XDEBUG_SESSION cookie set.

I’m assuming you have PhpStorm and Xdebug set up neat and nicely, as there are enough tutorials about it out there.

This works nicely when I start a debug session using the browser, but now we don’t use a browser and send the requests via HTTP directly (Codeception uses Guzzle for that), so we have to set the Cookie somehow. There are (at least) three ways to do that with Codeception:

  1. Set the cookie manually in the Cest (Codeception acceptance test class)
    $I->setCookie('XDEBUG_SESSION', 'PHPSTORM');
    
  2. Configure Guzzle to send the Cookie via the codeception.yml

    This is described Codeception manual on the PhpBrowser module

  3. Use the codeception/remote-debug extension

    Codeception-Remotedebug on Github

    This is my preferred way, it’s actually pretty easy to set up, and this way I just send the Cookie all the time.

    extensions:
      enabled:
        - Codeception\Extension\RemoteDebug
        config:
          Codeception\Extension\RemoteDebug:
            sessionName: PHPSTORM
    
    Since the Cookie value (PHPSTORM) might be different for other developers, these settings should not be under Version control. Codeception allows several ways to achieve this, in our project we use a environment-specific setup, which is not under version control and allows use to extend or overwrite settings on the project-wide, version controlled codeception.yml. Learn more about it in the Codeception manual on Environments.

    Increase PhpStorms Max. connections

    This might cause some headache if you forget about it. When I tried to all of the above ways to set the Cookie, the debug session was always getting stuck, i.e. PhpStorm was not stopping at the breakpoints and rather did – nothing!

    phpstorm_debug_max_connections

    By default, PhpStorm is configured to handle 1 debug session at a time. However in my case, I need to handle 2 debug sessions in parallel now – one debug session for the actual Acceptance test, and one for the API.

    And that’s it. I hope this will be useful for other Codeception users as well!

Filesystem abstraction in PHP, and why it sometimes is useful!

Phase 1: Ignorance and Bias

Recently I came across Flysystem, which is a Filesystem abstraction layer for PHP. Now if you haven’t worked with a Filesystem abstraction layer before (like me), you may ask yourself:

Why on earth should I add even more overhead for something basic as the file system?

Let me explain why it will be extremely useful. Say we have to build a small app in PHP which should download files from an FTP server, process them in some way, and store the result on the servers filesystem. No doubt, this can be done with PHPs build-in functions without a problem, you might say. Sure, but don’t forget the Unit Tes… dang! See?

Phase 2: Skepticism and Curiosity

The solution for our problem once more is: Abstraction. Abstract the process from any file system. Just use the same set of operations on some sort of adapter, no matter where or how the files are stored, and let the Abstraction Layer do the rest. You guessed it already, this is where Flysystem comes into play! Let’s start with a simple file download:

<?php

namespace My\Cool\Project;

use League\Flysystem\FilesystemInterface;

class Downloader
{
    /**
     * @var FilesystemInterface
     */
    protected $remoteFileSystem;

    /**
     * @var FilesystemInterface
     */
    protected $localFileSystem;

    /**
     * @param FilesystemInterface $remoteFileSystem
     * @param FilesystemInterface $localFileSystem
     */
    public function __construct(
        FilesystemInterface $remoteFileSystem,
        FilesystemInterface $localFileSystem
    ) {
        $this->remoteFileSystem = $remoteFileSystem;
        $this->localFileSystem = $localFileSystem;
    }

    /**
     * Download files from the remote file system
     * @param string $fileName The filename
     * @return bool
     */
    public function download($fileName)
    {
        $fileDownloaded = false;
        
        // This is the actual download part!
        if (!$this->localFileSystem->has($fileName)) {
            $fileDownloaded = $this->localFileSystem->writeStream(
                $fileName,
                $this->remoteFileSystem->readStream($fileName)
            );
        }
        // Done!

        return $fileDownloaded;
    }
}

Is that it?? Aww yeah, that’s it. When it’s with Flysystem, we only need a few lines of code, because it’s so intense!

To make the code it more clear, I called the filesystems $localFileSystem and $remoteFileSystem, but of course both filesystems could be remote ones, e.g. FTP and Amazon S3. This means that by using Flysystem, you completely decouple your code from the filesystem, so you’re prepared when the Business suddenly decides to “implement some totally easy changes”.

Of course our example is not 100%ly done, since we still have to create the Filesystem adapters we pass upon instantiation. But that’s ok, we need to do the configuration somewhere anyway. Below function is part of another class, which makes use of our Downloader class:

public function downloadSomeFile()
{
    $ftpAdapter = new \League\Flysystem\Adapter\Ftp([
        'host' => 'ftp.server.com',
        'username' => '123',
        'password' => 'abc'
    ]);
    $remoteFileSystem = new \League\Flysystem\Filesystem($ftpAdapter);

    $localAdapter = new \League\Flysystem\Adapter\Local('/tmp');
    $localFileSystem = new \League\Flysystem\Filesystem($localAdapter);

    $downloader = new \My\Cool\Project\Downloader(
        $remoteFileSystem, 
        $localFileSystem
    );
    
    $downloader->download('somefile.txt');
}

This is what I also like about using Flysystem: within the Downloader class we don’t have to deal with any path settings or such. It is completely done within the code which instantiates the Downloader, i.e. in the domain where it belongs.

it's so decoupled

Phase 3: Awesomeness and Why-didn’t-I-use-it-before-ness

I assume most of you didn’t even read until here, since you’ve been totally blown away and immediately started refactoring your code, giggling slightly insane. But wait, we’re not even fully through yet. Remember the question about Unit Tests? With Flysystem, it’s a breeze:

public function testCanDownloadFileFromRemoteFileSystem()
{
    // This mocks the remote server (could e.g. be an FTP) where we want to 
    // download the file from
    $remoteAdapter = new \League\Flysystem\Vfs\VfsAdapter(new Vfs);
    $remoteFileSystem = new \League\Flysystem\Filesystem($remoteAdapter);

    // Put a file on the virtual remote filesystem
    $remoteFileSystem->put('somefile.txt', 'just some content, does not matter');

    // Virtualize our local file system also, so no need to clean up or anything, 
    // since by using vfsStream, we don't even have to take care where to store 
    // the files during our tests.
    $localAdapter = new \League\Flysystem\Vfs\VfsAdapter(new Vfs);
    $localFileSystem = new \League\Flysystem\Filesystem($localAdapter);

    $downloader = new \My\Cool\Project\Downloader(
        $remoteFileSystem, 
        $localFileSystem
    );

    $this->assertTrue($downloader->download('somefile.txt'));
    $this->assertTrue($localFileSystem->has('somefile.txt'));
}

Flysystem even provides an adapter for vfsStream, a stream wrapper for PHP which helps greatly when you have to test filesystem-related functionality.

In simple terms, it simulates a file system in memory, which many (but not all) PHP functions can use.

Check it out here if you don’t know it yet: vfsStream on Github

So here we are: we just wrote a test for the FTP download in 5 minutes – without having to change a single line in the Downloader, without relying on a real FTP server. We could exchange the remote filesystem with Dropbox or even your Grandma’s closet within two minutes. Awesome. Now get a coffee and head over to your co-worker, who’s still struggling with the Downloader class and PHPs lovely native FTP support.

Local composer package development

A new feature on composer which I really appreciate is the “Path” repository type.

Imagine you are working on your super-cool-project, and one task requires new functionality where there is no composer package existing yet – yeah, this probably is a bit hard to imagine. Now you want to create your own super-awesome-package, because we all like it nice and separated and such.

So far, I would have developed the super-awesome-package directly inside the vendor folder of my super-cool-project – which somehow did not feel right. Obviously others felt the same, so now we can use the “path” resource type for packages which you develop right at the moment!

{
  "require": {
    "my-namespace/shoppingcart": "*@dev"
  },
  "repositories": [
    {
      "type": "path",
      "url": "../shoppingcart"
    }
  ]
}

Instead of using VCS as package source, you simply specify the directory (relative or absolute path both work) where your super-cool-package is located on your machine. In my case, I just refer to the shoppingcart folder, where the package is currently developed.

Composer will then add a symlink in the vendor folder upon running composer update, so all your changes on that package can immediately be used in your super-cool-project.

Now that’s what I call convenient!

Updating Typo3 CMS 6.2 to 7.2

Updating our companies major portals from Typo3 CMS 4.7 LTS to 6.2 LTS was a good piece of work, so I was not really keen on updating any websites again. A good excuse for us was the fact that RealURL was not yet available for Typo3 CMS 7, but now it is – no more excuses. We need to do it anyway.

Before you even start, you should check the requirements of Typo3 CMS 7. It requires at least PHP 5.5 and MySQL 5.5, which might not be available on your server. So check the Typo3 CMS 7 System requirements before you continue.

I started with a relatively small project, which was ideal to get some experience. Only a few dozens of pages, a couple of TypoScript files, and most important, no legacy extensions but only good maintained Extbase 6.2 extension. I was pleasently surprised that the whole migration with fixing all issues (see below) only took roughly one hour, so my thanks to the Typo3 CMS team for that great job!

If you have one or more pre Extbase 6.2 extensions which don’t yet use namespaces (i.e. classes are named like Tx_MyExtension_Domain_Model_Whatever instead of just Whatever), there is an extension called “compatibility6” which comes bundled with Version 7.2. You can also use it if you already screwed your Typo3 installation and you can’t login into the backend anymore: you can install it manually by editing the typo3conf/PackageStates.php, search for compatibility6 and change the state from inactive to active. Please note that the usage of this extension has a negative impact on performance, although I can’t quantify the impact. It might just be ok for smaller websites.

The update process is explained here: http://wiki.typo3.org/TYPO3_CMS_7.2.0. In our case, the basic steps were:

  • download the Typo3 sources
  • uninstall RealURL
  • delete typo3temp/Cache folder
  • switch typo3_src to new version
  • run all checks in the Install tool
  • login into the backend, update RealURL to the recent version and
  • install it again

Update problems

Of course the update did not go without any problems, so let’s discuss them in this section.

CSS missing!

After finishing the update work in the Install tool I went to the backend login page and was somewhat suprised – I didn’t expect that fresh oldschool style:

Typo3_login_CSS_problems

Ok wait, there’s something wrong. Obviously the t3skin.css was missing.

It turned out that somehow the PackageStates.php was mixed up, and although the t3skin system extension was marked as active, it obviously wasn’t. Doing some internet research revealed that this problem occurred for others as well. I ended up in simply deleting the PackageStates.php (after doing a backup) and reload the login page. This way, the PackageStates.php was regenerated from scratch. Once it was created again, I needed to re-activate almost all system extensions manually. All in all this was just a matter of minutes, so no big deal.

No leading backslashes please!

After fixing the CSS issues, I loaded the start page which includes one of our custom extensions, and ended up with another exception:

$className “\My\Nifty\Class” must not start with a backslash.

Some of our makeInstance() calls had a backslash before the FQCN, which whith Typo3 CMS 7 is not allowed anymore. Since the backslash is not needed, we simply removed it, e.g.

// works ok with Typo3 CMS 6.2, but not 7.2:
$myInstance = GeneralUtility::makeInstance('\\My\\Nifty\\Class');

// works ok with Typo3 CM 6.2 AND 7.2:
$myInstance = GeneralUtility::makeInstance('My\\Nifty\\Class');

This also applies for calls to the ObjectManagers get-function!

Stating the class name as a string leads to the ugly double backslashes, which nobody likes. With PHP 5.5+, we now can use the ::class constant:

$myInstance = GeneralUtility::makeInstance(My\Nifty\Class:class);

which I really prefer because it is more readable and you can now use your IDEs code completion.

When the views are gone

Next reload, the page loads … yay! No exceptions. Good. So on to the next issue (a real classic):

Sorry, the requested view was not found. 

The technical reason is: No template was found. View could not be resolved for action "index" in class "My\Nifty\Controller\IndexController".

This time the problem was in the Configuration\Typoscript\setup.txt of one extension, where we still used the old name for the view folders:

# this works until Typo3 6.2
templateRootPath = {$plugin.tx_mynifty.view.templateRootPath}
partialRootPath = {$plugin.tx_mynifty.view.partialRootPath}
layoutRootPath = {$plugin.tx_mynifty.view.layoutRootPath}

# with Typo3 7.x, use
templateRootPaths.10 = {$plugin.tx_mynifty.view.templateRootPath}
partialRootPaths.10 = {$plugin.tx_mynifty.view.partialRootPath}
layoutRootPaths.10 = {$plugin.tx_mynifty.view.layoutRootPath}

Extbase supports fallback paths for templates, layouts and partials, which is quite cool – when you know it. Fluid uses the highest index (in our case only one, the 10) as directory for looking up the templates, and tries out the next lower one if it fails. Oh, and also it is called RootPaths instead of RootPath for a while already.

Say “hi” to number seven

Updating to Typo3 CMS 7.2 caused far less trouble than the previous “LTS-Jump”. I really like the fresh look of the backend, and the fact that the Typo3 core has been cleaned up even further.

Surely the website I migrated was only a small one with no legacy extensions, but I think I covered some issues which might be helpful to others!

Typo3 CMS hooks and namespaced classes

During work on my extension cwenvbanner to make it compatible with Typo3 CMS 7.2, I came across a good old friend, the hooks array $GLOBALS[‘TYPO3_CONF_VARS’][‘SC_OPTIONS’]. Since I sometimes struggle with the way how to declare the class and function to be called, I decided to write it down here – once and for all 🙂

In short, this is how a hook can be defined

EXT:name_of_extension/path_and_filename_of_class:{&}classname->functionName

The ampersand before the class name is optional. If you add it, the class will be instantiated using the Singleton pattern.

Let’s have some examples. Firstly, let’s have a look at the old version I had in my extension:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['contentPostProc-output']['tx_cwenvbanner']
	= 'EXT:cwenvbanner/class.tx_cwenvbanner.php:&tx_cwenvbanner->contentPostProc_output';

For the new Typo3 CMS 7 compatible version of cwenvbanner I wanted to use a namespaced classes for the heck of it. I came up with the inspiring name “Main” which now resides in the Classes directory of the extension, so need I changed the path and filename accordingly. I also need to provide the fully qualified class name, i.e. Cw\Cwenvbanner\Main.

This is how it looks like now:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['contentPostProc-output']['tx_cwenvbanner']
	= 'EXT:cwenvbanner/Classes/Main.php:&Cw\\Cwenvbanner\\Main->contentPostProc_output';

Fixing Locale issues on Debian

Business as usual. New feature developed, tested, packed and ready to go. Let me just deploy it to production, smoke test aaaand …. damn! What happened to the currency symbols!?

In my particular case, PHP’s setlocale() returned false – so obviously those darned locales stroke again. SearchEngineOfYourChoice(TM) to the rescue!

Luckily even I was ably to fix this problem quickly without needing to contact the Admins (who wouldn’t have been in the office anyway):

$ locale -a
C
C.UTF-8
en_US.utf8
POSIX

Aha! It was missing almost all locales. However adding them is quite easy:

$ sudo dpkg-reconfigure locales

Generating locales (this might take a while)...
de_DE.ISO-8859-1... done
de_DE.UTF-8... done
de_DE.ISO-8859-15@euro... done
en_US.UTF-8... done
fr_FR.ISO-8859-1... done
fr_FR.UTF-8... done
Generation complete.

The dialog that appears after running this command shouldn’t be a problem at all, so I don’t mention it further here. Just select the locales you need there and go for it.

Afterwards check the results by again doing

$ locale -a

C
C.UTF-8
de_DE
de_DE@euro
de_DE.iso88591
de_DE.iso885915@euro
de_DE.utf8
deutsch
en_US.utf8
français
french
fr_FR
fr_FR.iso88591
fr_FR.utf8
german
POSIX

That’s already it. Don’t forget to restart your webserver!