cwmobileredirect and cwenvbanner finally available for Typo3 7 LTS

Well, actually the Typo3 7 LTS versions were available for a couple of months through my Github repositories of cwmobileredirect and cwenvbanner already, however I never found the time to upload them to TER. Finally I found some time to upload it, so here you go:

cwmobileredirect on TER
cwenvbanner on TER

The new versions do not include any new features, I did some cleanup though, so Typo3 4.x is no longer supported. Please just stick with the old versions of the extensions in case you are still using one of these ancient Typo3 installations.

Besides Typo3 7 compatibility and cleanup I spend some time on making them work with a Composer based Typo3 installation (the outcome can be found on Github as well) and Travis CI. Since I only work on these extensions occasionally, this will surely help me in with the Typo3 8 LTS versions (which I hope to get done a bit earlier this time :-)).

Updating Typo3 CMS 6.2 to 7.2

Updating our companies major portals from Typo3 CMS 4.7 LTS to 6.2 LTS was a good piece of work, so I was not really keen on updating any websites again. A good excuse for us was the fact that RealURL was not yet available for Typo3 CMS 7, but now it is – no more excuses. We need to do it anyway.

Before you even start, you should check the requirements of Typo3 CMS 7. It requires at least PHP 5.5 and MySQL 5.5, which might not be available on your server. So check the Typo3 CMS 7 System requirements before you continue.

I started with a relatively small project, which was ideal to get some experience. Only a few dozens of pages, a couple of TypoScript files, and most important, no legacy extensions but only good maintained Extbase 6.2 extension. I was pleasently surprised that the whole migration with fixing all issues (see below) only took roughly one hour, so my thanks to the Typo3 CMS team for that great job!

If you have one or more pre Extbase 6.2 extensions which don’t yet use namespaces (i.e. classes are named like Tx_MyExtension_Domain_Model_Whatever instead of just Whatever), there is an extension called “compatibility6” which comes bundled with Version 7.2. You can also use it if you already screwed your Typo3 installation and you can’t login into the backend anymore: you can install it manually by editing the typo3conf/PackageStates.php, search for compatibility6 and change the state from inactive to active. Please note that the usage of this extension has a negative impact on performance, although I can’t quantify the impact. It might just be ok for smaller websites.

The update process is explained here: http://wiki.typo3.org/TYPO3_CMS_7.2.0. In our case, the basic steps were:

  • download the Typo3 sources
  • uninstall RealURL
  • delete typo3temp/Cache folder
  • switch typo3_src to new version
  • run all checks in the Install tool
  • login into the backend, update RealURL to the recent version and
  • install it again

Update problems

Of course the update did not go without any problems, so let’s discuss them in this section.

CSS missing!

After finishing the update work in the Install tool I went to the backend login page and was somewhat suprised – I didn’t expect that fresh oldschool style:

Typo3_login_CSS_problems

Ok wait, there’s something wrong. Obviously the t3skin.css was missing.

It turned out that somehow the PackageStates.php was mixed up, and although the t3skin system extension was marked as active, it obviously wasn’t. Doing some internet research revealed that this problem occurred for others as well. I ended up in simply deleting the PackageStates.php (after doing a backup) and reload the login page. This way, the PackageStates.php was regenerated from scratch. Once it was created again, I needed to re-activate almost all system extensions manually. All in all this was just a matter of minutes, so no big deal.

No leading backslashes please!

After fixing the CSS issues, I loaded the start page which includes one of our custom extensions, and ended up with another exception:

$className “\My\Nifty\Class” must not start with a backslash.

Some of our makeInstance() calls had a backslash before the FQCN, which whith Typo3 CMS 7 is not allowed anymore. Since the backslash is not needed, we simply removed it, e.g.

// works ok with Typo3 CMS 6.2, but not 7.2:
$myInstance = GeneralUtility::makeInstance('\\My\\Nifty\\Class');

// works ok with Typo3 CM 6.2 AND 7.2:
$myInstance = GeneralUtility::makeInstance('My\\Nifty\\Class');

This also applies for calls to the ObjectManagers get-function!

Stating the class name as a string leads to the ugly double backslashes, which nobody likes. With PHP 5.5+, we now can use the ::class constant:

$myInstance = GeneralUtility::makeInstance(My\Nifty\Class:class);

which I really prefer because it is more readable and you can now use your IDEs code completion.

When the views are gone

Next reload, the page loads … yay! No exceptions. Good. So on to the next issue (a real classic):

Sorry, the requested view was not found. 

The technical reason is: No template was found. View could not be resolved for action "index" in class "My\Nifty\Controller\IndexController".

This time the problem was in the Configuration\Typoscript\setup.txt of one extension, where we still used the old name for the view folders:

# this works until Typo3 6.2
templateRootPath = {$plugin.tx_mynifty.view.templateRootPath}
partialRootPath = {$plugin.tx_mynifty.view.partialRootPath}
layoutRootPath = {$plugin.tx_mynifty.view.layoutRootPath}

# with Typo3 7.x, use
templateRootPaths.10 = {$plugin.tx_mynifty.view.templateRootPath}
partialRootPaths.10 = {$plugin.tx_mynifty.view.partialRootPath}
layoutRootPaths.10 = {$plugin.tx_mynifty.view.layoutRootPath}

Extbase supports fallback paths for templates, layouts and partials, which is quite cool – when you know it. Fluid uses the highest index (in our case only one, the 10) as directory for looking up the templates, and tries out the next lower one if it fails. Oh, and also it is called RootPaths instead of RootPath for a while already.

Say “hi” to number seven

Updating to Typo3 CMS 7.2 caused far less trouble than the previous “LTS-Jump”. I really like the fresh look of the backend, and the fact that the Typo3 core has been cleaned up even further.

Surely the website I migrated was only a small one with no legacy extensions, but I think I covered some issues which might be helpful to others!

Typo3 CMS hooks and namespaced classes

During work on my extension cwenvbanner to make it compatible with Typo3 CMS 7.2, I came across a good old friend, the hooks array $GLOBALS[‘TYPO3_CONF_VARS’][‘SC_OPTIONS’]. Since I sometimes struggle with the way how to declare the class and function to be called, I decided to write it down here – once and for all 🙂

In short, this is how a hook can be defined

EXT:name_of_extension/path_and_filename_of_class:{&}classname->functionName

The ampersand before the class name is optional. If you add it, the class will be instantiated using the Singleton pattern.

Let’s have some examples. Firstly, let’s have a look at the old version I had in my extension:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['contentPostProc-output']['tx_cwenvbanner']
	= 'EXT:cwenvbanner/class.tx_cwenvbanner.php:&tx_cwenvbanner->contentPostProc_output';

For the new Typo3 CMS 7 compatible version of cwenvbanner I wanted to use a namespaced classes for the heck of it. I came up with the inspiring name “Main” which now resides in the Classes directory of the extension, so need I changed the path and filename accordingly. I also need to provide the fully qualified class name, i.e. Cw\Cwenvbanner\Main.

This is how it looks like now:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['contentPostProc-output']['tx_cwenvbanner']
	= 'EXT:cwenvbanner/Classes/Main.php:&Cw\\Cwenvbanner\\Main->contentPostProc_output';

New Typo3 extension unleashed: cwdblog

Working with Extbase a lot, I always found it very cumbersome to see the actual SQL queries which were produced by the Persistance layer. One solution would be to turn on MySQLs general log, but this is not very convenient.

So I decided to change that. The result can be found here: cwdblog on GitHub

cwdblog will log all DB queries which are run against the table(s) you can configure via the Extension Manager. It will be logged using the system log and the devlog extension (if installed). It will also write the queries into a global array (GLOBALS[‘___cwdblog’]) so that it is even easier to access to debug sessions.

In order to tell cwdblog which tables to actually log, please go to the extension manager, edit the extension configuration and change the tableName regular expression as you need.

Slashes will be added, so e.g. mycool will result in /mycool/, which will let cwdblog log all queries for tables which include the string ‘mycool’. This could be e.g. tx_mycoolextension_domain_model_blog or tx_myotherextension_domain_model_mycooltable.

Please note that this extension should not be used on production servers yet, since it is not very well tested).

Of Typo3, Site Crawlers and Compression

A Typo3-Installation I currently maintain uses the sitecrawler extension to heat up the page cache every morning before the users are visiting our site. We encountered the problem that all pages that were cached didn’t use gzipped CSS & JavaScript-Files, only the non gzipped versions.

Typo3 first generates both the gzipped and non-gzipped versions of the CSS & JS files, and then checks the HTTP_ACCEPT_ENCODING setting whether gzip is supported and decides which of both versions is referenced in the HTML.

Since the Crawler is not sending the HTTP_ACCEPT_ENCODING flag when crawling the pages, Typo3 thus renders the Page HTML referencing the non-gzipped files. I tried to add the “Accept Encoding” to the crawlers requestUrl()-Function, but of course Typo3 now returned the HTML as gzip, which totally broke the crawlers logic …

Since as said Typo3 generates both versions of the files, eventually we came up with these two redirects:

RewriteCond %{HTTP:Accept-Encoding} .*gzip.*
RewriteRule ^typo3temp/compressor/(.*)\.js$ typo3temp/compressor/$1.js.gzip?%{QUERY_STRING} [L]
RewriteCond %{HTTP:Accept-Encoding} .*gzip.*
RewriteRule ^typo3temp/compressor/(.*)\.css$ typo3temp/compressor/$1.css.gzip?%{QUERY_STRING} [L]

In case the page is cached without referencing to the gzipped versions (which is the default state every morning), we just redirect all request to the gzipped versions.

Traits and Extbase

With Extbase, accessing the extensions settings outside Controllers (e.g. in a Repository) requires some manual work which is always the same. I don’t know why Extbase doesn’t support us with helper functions, but anyway, let’s put this needed code where we can reuse it easily.

Being limited to PHP 5.3 until last month, my usual approach was to inherit the Repository from an Abstract class which contained a set of often used functions. This still works fine, but if you want these functions to be used in let’s say a Typo3 hook class, or a Scheduler task (where the Extbase support is even worse! Geez!), you probably end up writing a couple of Abstract classes which are more or less the same. Of course you can inherit the Abstract class from another abstract class … which again doesn’t work with Scheduler tasks, because they need to inherit from a base class anyways… nah…

Since we now use PHP 5.4 on our production servers (and yes, I am aware that 2014 is almost over), I felt urged to find out whether Traits can be used with Extbase. Surprisingly (working with Extbase and Typo3 is easier with a good amount of sarcasm) it was as easy as it should be! I tested it on Extbase 6.2, and the Autoloader loads the Trait without any hassle.

Here’s some example. First, the trait itself:

<?php

namespace Vendor\Myextension\Library;

trait Settings
{
    /**
     * The extensions name
     * @var string
     */
    protected $extensionName;

    /**
     * Typo3 Settings array
     * @var array
     */
    protected $settings;

    /**
     * @var \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface
     */
    protected $configurationManager;

    /**
     * @param \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface $configurationManager
     * @inject
     */
    public function injectConfigurationManager(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface $configurationManager) {
        $this->configurationManager = $configurationManager;
    }

    /**
     * Getter for $extensionName
     * @return string
     */
    public function getExtensionName() {
        if ($this->extensionName === NULL) {
            $className = get_class($this);
            if (strpos($className, '\\') !== FALSE) {
                $classNameParts = explode('\\', $className, 4);
                // Skip vendor and product name for core classes
                if (strpos($className, 'TYPO3\\CMS\\') === 0) {
                    $this->extensionName = $classNameParts[2];
                } else {
                    $this->extensionName = $classNameParts[1];
                }
            } else {
                list(, $this->extensionName) = explode('_', $className);
            }
        }

        return $this->extensionName;
    }

    /**
     * Setter for $extensionName
     * @param string $extensionName
     * @return Settings
     */
    public function setExtensionName($extensionName) {
        $this->extensionName = $extensionName;
        return $this;
    }

    /**
     * Setter for $settings
     * @param array $settings
     * @return $this
     */
    public function setSettings($settings) {
        $this->settings = $settings;

        return $this;
    }

    /**
     * Getter for $settings
     * The settings are loaded from the configurationManager on demand only.
     * If this is called outside Extbase (i.e. from a system hook, or a scheduler task) you must use
     * \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FULL_TYPOSCRIPT
     * and fetch the extensions settings via the array structure, e.g.
     * $this->settings['plugin.']['yourextension.']['persistence.']['storagePid']
     * @param string $settingsType
     * @return array
     */
    public function getSettings($settingsType = \TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_SETTINGS) {
        if ($this->settings === NULL) {
            $this->settings = $this->configurationManager->getConfiguration(
                $settingsType, $this->getExtensionName()
            );
        }

        return $this->settings;
    }
}

No rocket science, but stuff you need all over again. I decided to use “Library” as namespace, but this doesn’t really matter.

So far, so easy. To include a trait, you only need to add the “use”-statement inside your class, e.g. in a repository:

<?php

namespace Vendor\Myextension\Domain\Repository;

class SomeSeriousStuff
{
    // Note the "use" statement within the class. This inserts our Trait class.
    use \Vendor\Myextension\Library\Settings;

    public function mySeriousFunction() {
        $settings = $this->getSettings();
        // ... do some serious things here with the settings
    }
}

This works really smooth, I think I will get used to Traits very soon.

Fun fact: shortly after finishing my Extension where I used Traits for the first time, I was asked for a PHP 5.3 compatible version. Fuuuuuu……

Typo3 Caching and TypoScript conditions

Having taken over a somewhat shabby Typo3 project recently (I prefer using the term “challenge” here), digging through the many TypoScript files revealed something which is probably very common in many Typo3 setups:

[browser = msie]
page.includeCSS.file22 = fileadmin/templates/css/ie.css
[browser = msie] AND [version = 7]
page.includeCSS.file23 = fileadmin/templates/css/ie7.css
[GLOBAL]

Why is that a problem? Well, when it comes to caching, Typo3 starts building up Cache entries for every possible condition. So from the above example, we will get 3 cache entries:

  1. Internet Explorer 7
  2. Other Internet Explorers
  3. All other browsers except Internet Explorer

Of course this blows up the database, but space wasn’t the main problem in our case – it was the time it took to rebuild the cache after clearing it. This Typo3 installation serves thousands of pages, and they are crawled regularly to build up the cache, so we really don’t want to waste any time.

So what do we do? First and obvious step, drop IE 7 support! Ahhh, that feels so right…. Ok. Now the remaining, really few IE hacks in the ie.css will be included by using good old Conditional comments:

page.headerData.123 = TEXT
page.headerData.123.value (
<!--[if IE]>
<script type="text/css" src="fileadmin/templates/css/ie.css"></script>
<![endif]-->
)

The additional load for IE users is minimal, and we got rid of quite some cache entries and saved a good amount of computational power.

Template Engines vs. PHTML

The Problem: Performance. Once again.

It was time to think again about the use of Template Engines (i.e. those like Smarty or Fluid who offer their own syntax) versus so called PHTML files (i.e. basically mixed HTML and PHP), as one of my colleagues currently works on a Typo3 Extbase extension with a very huge HTML output. Unfortunately the rendering process was extremely slow. In our case Fluid, the template engine that comes bundled with Extbase, was to blame.

Although using compiled templates, it took about 6 seconds to render. After clearing the cache it was way beyond 20 seconds until the templates were compiled again. What the heck. Where does this massive overhead come from?

Looking at the compiled templates, I was a bit shocked: one of it had almost 20.000 lines of PHP code, tons of closures and function calls! It was just a template, now it is a monster. 6 seconds rendering time is not acceptable at all. Surely the extension produced a big HTML code, using loads of loops, partials and view helpers, but we had to find a way to deal with it.

Two options where obvious at that moment:

  1. Do we really need to use Fluid?
  2. How about loading at least parts of the content, which is not immediately used, via AJAX?

I admit I’m biased. I never liked using any Template Engine in PHP. That’s why I had a closer look at option 1, while my colleague started working on option 2 meanwhile.

Considerations

Let’s think about the Pros and Cons of using template engines in PHP. Here is what we came out with:

Pros:

  • No PHP knowledge required to build templates: yeah, sure. This is a common argument. So it is ok for the Webdesigner to learn a new template engine, but too much to learn PHP basics. Plus, at least in our team there is nobody who doesn’t speak PHP.
  • Templates are cleaner, easier to read: this depends on the template engine of course, but considering Fluid I would agree on that. It is not very important for me, though.
  • No risk of PHP abuse in template: Yip, good point. I guess we’ve all had these “WTF!?!” moments when we looked at PHTML templates, finding functions with business logic in templates. This surely requires the coders to be disciplined, and/or Code Reviews.
  • Template Engines offer caching mechanisms: true. But without caching they wouldn’t even come close to the performance of PHTML files. There’s nothing to say against implementing your own caching when you use PHTML templates, though.

Cons:

  • Using a template engine produces overhead: you can’t ignore the fact that a templating engine per se is additional overhead. This is not necessarily bad, when the overhead is low, but when the content takes more than double the time to render it might not be the correct choice.
  • Templates can not be debugged properly: I’m used to develop using a PHP debugger a lot, especially when I have to deal with many variables and nested arrays. I can’t use that in templates. I can use it in PHTML files.
  • “Why did my change don’t come through? Ah, I forgot to clear the cache again!”: If this applies to you, as it does for me, then template engines can be a bit of a pain sometimes.
  • “I’m not slacking off, my code’s compiling”: Not only a smart reference to XKCD, but also a very annoying aspect: if you develop, you either turn off caching completely (slow!), or you have to clear the cache whenever needed (annoying – and slow!).

Ok, there are some points which speak for Template Engines, but are they really worth the overhead? This surely depends on personal preferences, but for me it was clear: No. It is not.

Conclusion: Go back

Great. We wanted to stick with Extbase, but we didn’t want to use Fluid all the time. So we need a PHTML template renderer for Extbase. It should at least cover the following requirements:

  • We want true MVC! View and Logic must be separated.
  • Extbase/Fluid must not be modified. Copy&Paste has to be avoided.
  • The PHTML renderer should be easy to integrate. Existing controller actions shouldn’t be rewritten at all.
  • The use of Fluid should still be possible.
  • Partials must be supported.
  • Fluid ViewHelpers should be supported, at least some sort of ViewHelpers must exist. Namespaces for ViewHelpers would be nice.

Of course there is a mayor drawback: all the Fluid templates which already existed need to be “backported” to PHTML. Oh well…

It should however show that the required goal was very easy to achieve. This will be shown in the next post.

What are your views on Template Engines? Maybe we missed the one argument to use them instead of PHTML?