Debugging Flow3 – ain’t got time for that!?

It is known that XDebug has a negative impact on performance, but what happens when you try to debug a Flow3 application can be something between absolutely surprising and totally annoying. But one after another.

XDebug is my every day tool. I use it not only for debugging, but for regular development as well. I simply enjoy the benefit to always know which variables are available in the current scope (Typo3 globals, anyone?), I don’t need to recall nested array structures, I can evaluate my code on the fly and so on. So naturally, on my dev XDebug is always enabled and listening, both for PHP SAPI or Webserver.

Working on my first Flow3 project, I suddenly found myself waiting over 30 minutes for the Cache to rebuild after flushing it manually. Running the application on a different machine worked ok, so it was obvious that the issue was somewhere in my setup. Wtf?

So I did what a normal XDebug user would do – let’s profile that damn thing! I realized the core problem as I found dozends of sometimes quite huge Cachegrind files in my output directory. What the heck, there must be really some serious action going on (and it does – see box below)! So I disabled XDebug for the moment, and voilà – the Cache was rebuild within seconds.

If you have not worked with Flow3 before you have to know that Flow3 actually requires cached class files because of the use of Aspect-oriented programming (AOP). In short, Flow3 scans the code files of the packages whether certain rules (the so called Aspects, defined via Annotations) apply, and changes the code in the classes according to the Aspect. Doing this for every request would be obviously insane, so the results are cached. We suddenly find our good old PHP scripts in some way being compiled.

There is a nice article about Flow3 and AOP to be found here which describes it more in detail.

Flow3 luckily has some advanced file monitoring which, when you work in Development context, makes it unnecessary to clear the whole cache manually every time you change a class, and rebuilding the cache for one class only will work within a reasonable amount of time even with XDebug turned on. Nevertheless sometimes you will need to flush the whole cache, and then you’re doomed. En-/Disabling XDebug in the php.ini manually would be too cumbersome (for me). So there has to be a more convenient solution.

How to debug Flow3 applications and stay sane

Here’s with what I ended up: I disabled XDebug for PHP via CLI by default, and enable it only when I need it using the “php -d” switch:

php -dxdebug.remote_autostart=On -dxdebug.remote_enable=On ./flow

Not a very easy one to remember, but we’re getting close. Why not put this into a separate shell script? Yes, why not. I called it “flowdebug”, and it is located in the project root (where the original flow command is located).

#!/bin/sh
export PHP_IDE_CONFIG="serverName=my.server"
export XDEBUG_CONFIG="idekey=PHPSTORM"
php -dxdebug.remote_autostart=On -dxdebug.remote_enable=On ./flow "$@"

As you can see I also put the environment variables needed to start the debugging session in that script.

So from now on, when I want to debug my Flow3 application, I simply use

./flowdebug mypackage:mycommand

Traits and Extbase

With Extbase, accessing the extensions settings outside Controllers (e.g. in a Repository) requires some manual work which is always the same. I don’t know why Extbase doesn’t support us with helper functions, but anyway, let’s put this needed code where we can reuse it easily.

Being limited to PHP 5.3 until last month, my usual approach was to inherit the Repository from an Abstract class which contained a set of often used functions. This still works fine, but if you want these functions to be used in let’s say a Typo3 hook class, or a Scheduler task (where the Extbase support is even worse! Geez!), you probably end up writing a couple of Abstract classes which are more or less the same. Of course you can inherit the Abstract class from another abstract class … which again doesn’t work with Scheduler tasks, because they need to inherit from a base class anyways… nah…

Since we now use PHP 5.4 on our production servers (and yes, I am aware that 2014 is almost over), I felt urged to find out whether Traits can be used with Extbase. Surprisingly (working with Extbase and Typo3 is easier with a good amount of sarcasm) it was as easy as it should be! I tested it on Extbase 6.2, and the Autoloader loads the Trait without any hassle.

Here’s some example. First, the trait itself:

<?php

namespace Vendor\Myextension\Library;

trait Settings
{
    /**
     * The extensions name
     * @var string
     */
    protected $extensionName;

    /**
     * Typo3 Settings array
     * @var array
     */
    protected $settings;

    /**
     * @var \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface
     */
    protected $configurationManager;

    /**
     * @param \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface $configurationManager
     * @inject
     */
    public function injectConfigurationManager(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface $configurationManager) {
        $this->configurationManager = $configurationManager;
    }

    /**
     * Getter for $extensionName
     * @return string
     */
    public function getExtensionName() {
        if ($this->extensionName === NULL) {
            $className = get_class($this);
            if (strpos($className, '\\') !== FALSE) {
                $classNameParts = explode('\\', $className, 4);
                // Skip vendor and product name for core classes
                if (strpos($className, 'TYPO3\\CMS\\') === 0) {
                    $this->extensionName = $classNameParts[2];
                } else {
                    $this->extensionName = $classNameParts[1];
                }
            } else {
                list(, $this->extensionName) = explode('_', $className);
            }
        }

        return $this->extensionName;
    }

    /**
     * Setter for $extensionName
     * @param string $extensionName
     * @return Settings
     */
    public function setExtensionName($extensionName) {
        $this->extensionName = $extensionName;
        return $this;
    }

    /**
     * Setter for $settings
     * @param array $settings
     * @return $this
     */
    public function setSettings($settings) {
        $this->settings = $settings;

        return $this;
    }

    /**
     * Getter for $settings
     * The settings are loaded from the configurationManager on demand only.
     * If this is called outside Extbase (i.e. from a system hook, or a scheduler task) you must use
     * \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FULL_TYPOSCRIPT
     * and fetch the extensions settings via the array structure, e.g.
     * $this->settings['plugin.']['yourextension.']['persistence.']['storagePid']
     * @param string $settingsType
     * @return array
     */
    public function getSettings($settingsType = \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_SETTINGS) {
        if ($this->settings === NULL) {
            $this->settings = $this->configurationManager->getConfiguration(
                $settingsType, $this->getExtensionName()
            );
        }

        return $this->settings;
    }
}

No rocket science, but stuff you need all over again. I decided to use “Library” as namespace, but this doesn’t really matter.

So far, so easy. To include a trait, you only need to add the “use”-statement inside your class, e.g. in a repository:

<?php

namespace Vendor\Myextension\Domain\Repository;

class SomeSeriousStuff
{
    // Note the "use" statement within the class. This inserts our Trait class.
    use \Vendor\Myextension\Library\Settings;

    public function mySeriousFunction() {
        $settings = $this->getSettings();
        // ... do some serious things here with the settings
    }
}

This works really smooth, I think I will get used to Traits very soon.

Fun fact: shortly after finishing my Extension where I used Traits for the first time, I was asked for a PHP 5.3 compatible version. Fuuuuuu……

Remote CLI debugging with XDebug

There are quite some tutorials out there on how to use XDebug to debug PHP CLI code, which where more confusing than helpful (to me). I found out that, once you have a running XDebug configuration, it is very easy to debug on the console, particularly with phpStorm.

Simply run the following line once

export PHP_IDE_CONFIG="serverName=www.domain.com" ; export XDEBUG_CONFIG="idekey=PHPSTORM"

(after customizing it to your needs, of course), set your breakpoints and run the script. That’s already it. You can also put this into your .bashrc file.

T3CON14

Currently I’m attending the 10th International Typo3 Conference (or in cool: T3CON14) in Berlin, and although it is not over yet I really liked it so far. Not only because of the great location in one of the hippest parts of Berlin, or the tasty food, or that I was able to carry home a 3-years supply of stylish pens, no … the atmosphere is fantastic and it is very nice to see the faces behind the names. Also many sessions where very inspirational. My favorite so far was the brilliant speech of Clemens Prerovsky (Aloha-Editor).

Also, and I’m definitely no fanboy, but to attend a speech of the Typo3-founder Kaspar Skårhøj was really something I would not have expected after all these years ;-)

Typo3 Caching and TypoScript conditions

Having taken over a somewhat shabby Typo3 project recently (I prefer using the term “challenge” here), digging through the many TypoScript files revealed something which is probably very common in many Typo3 setups:

[browser = msie]
page.includeCSS.file22 = fileadmin/templates/css/ie.css
[browser = msie] AND [version = 7]
page.includeCSS.file23 = fileadmin/templates/css/ie7.css
[GLOBAL]

Why is that a problem? Well, when it comes to caching, Typo3 starts building up Cache entries for every possible condition. So from the above example, we will get 3 cache entries:

  1. Internet Explorer 7
  2. Other Internet Explorers
  3. All other browsers except Internet Explorer

Of course this blows up the database, but space wasn’t the main problem in our case – it was the time it took to rebuild the cache after clearing it. This Typo3 installation serves thousands of pages, and they are crawled regularly to build up the cache, so we really don’t want to waste any time.

So what do we do? First and obvious step, drop IE 7 support! Ahhh, that feels so right…. Ok. Now the remaining, really few IE hacks in the ie.css will be included by using good old Conditional comments:

page.headerData.123 = TEXT
page.headerData.123.value (
<!--[if IE]>
<script type="text/css" src="fileadmin/templates/css/ie.css"></script>
<![endif]-->
)

The additional load for IE users is minimal, and we got rid of quite some cache entries and saved a good amount of computational power.

Use HeidiSQL with your vagrant box

I already discussed how to use HeidiSQL with a remote server in a former post. I now switched to use a local Vagrant VM instead of a remote VM. Of course I still wanted to use my beloved HeidiSQL. And it is – again – very easy!

vagrant_setup_1

On the above page, you need to use the MySQL Username and Password.

vagrant_setup_2

On the “SSH tunnel” tab we have to use the Username and Password for the SSH connection. I use a Private Key instead of a password, so I leave the password empty and specify the Private key file instead.

My box is configured to use SSH port 2222 instead of the default port 22. This will most likely differ on other boxes, so it is worth to check this first if the connection is not possible.

That’s it :-)

Template Engines vs. PHTML

The Problem: Performance. Once again.

It was time to think again about the use of Template Engines (i.e. those like Smarty or Fluid who offer their own syntax) versus so called PHTML files (i.e. basically mixed HTML and PHP), as one of my colleagues currently works on a Typo3 Extbase extension with a very huge HTML output. Unfortunately the rendering process was extremely slow. In our case Fluid, the template engine that comes bundled with Extbase, was to blame.

Although using compiled templates, it took about 6 seconds to render. After clearing the cache it was way beyond 20 seconds until the templates were compiled again. What the heck. Where does this massive overhead come from?

Looking at the compiled templates, I was a bit shocked: one of it had almost 20.000 lines of PHP code, tons of closures and function calls! It was just a template, now it is a monster. 6 seconds rendering time is not acceptable at all. Surely the extension produced a big HTML code, using loads of loops, partials and view helpers, but we had to find a way to deal with it.

Two options where obvious at that moment:

  1. Do we really need to use Fluid?
  2. How about loading at least parts of the content, which is not immediately used, via AJAX?

I admit I’m biased. I never liked using any Template Engine in PHP. That’s why I had a closer look at option 1, while my colleague started working on option 2 meanwhile.

Considerations

Let’s think about the Pros and Cons of using template engines in PHP. Here is what we came out with:

Pros:

  • No PHP knowledge required to build templates: yeah, sure. This is a common argument. So it is ok for the Webdesigner to learn a new template engine, but too much to learn PHP basics. Plus, at least in our team there is nobody who doesn’t speak PHP.
  • Templates are cleaner, easier to read: this depends on the template engine of course, but considering Fluid I would agree on that. It is not very important for me, though.
  • No risk of PHP abuse in template: Yip, good point. I guess we’ve all had these “WTF!?!” moments when we looked at PHTML templates, finding functions with business logic in templates. This surely requires the coders to be disciplined, and/or Code Reviews.
  • Template Engines offer caching mechanisms: true. But without caching they wouldn’t even come close to the performance of PHTML files. There’s nothing to say against implementing your own caching when you use PHTML templates, though.

Cons:

  • Using a template engine produces overhead: you can’t ignore the fact that a templating engine per se is additional overhead. This is not necessarily bad, when the overhead is low, but when the content takes more than double the time to render it might not be the correct choice.
  • Templates can not be debugged properly: I’m used to develop using a PHP debugger a lot, especially when I have to deal with many variables and nested arrays. I can’t use that in templates. I can use it in PHTML files.
  • “Why did my change don’t come through? Ah, I forgot to clear the cache again!”: If this applies to you, as it does for me, then template engines can be a bit of a pain sometimes.
  • “I’m not slacking off, my code’s compiling”: Not only a smart reference to XKCD, but also a very annoying aspect: if you develop, you either turn off caching completely (slow!), or you have to clear the cache whenever needed (annoying – and slow!).

Ok, there are some points which speak for Template Engines, but are they really worth the overhead? This surely depends on personal preferences, but for me it was clear: No. It is not.

Conclusion: Go back

Great. We wanted to stick with Extbase, but we didn’t want to use Fluid all the time. So we need a PHTML template renderer for Extbase. It should at least cover the following requirements:

  • We want true MVC! View and Logic must be separated.
  • Extbase/Fluid must not be modified. Copy&Paste has to be avoided.
  • The PHTML renderer should be easy to integrate. Existing controller actions shouldn’t be rewritten at all.
  • The use of Fluid should still be possible.
  • Partials must be supported.
  • Fluid ViewHelpers should be supported, at least some sort of ViewHelpers must exist. Namespaces for ViewHelpers would be nice.

Of course there is a mayor drawback: all the Fluid templates which already existed need to be “backported” to PHTML. Oh well…

It should however show that the required goal was very easy to achieve. This will be shown in the next post.

What are your views on Template Engines? Maybe we missed the one argument to use them instead of PHTML?

Free! JavaScript! Literature!

Back in the days, one of my profs at the University always had some cool quotes at hand. One of his favourites was “There is no such thing as a free lunch”. While I basically agree (at least what food is concerned), you often find free gems in the internet, like a series of JavaScript related books:

https://github.com/getify/You-Dont-Know-JS

They are available for free on github, however the author also sells them as e-book or even printed versions. I’m not sure if I would pay $17 for a 98 pages e-book which I can read for free, though… that sounds a bit too expensive in my opinion.