Filesystem abstraction in PHP, and why it sometimes is useful!

Phase 1: Ignorance and Bias

Recently I came across Flysystem, which is a Filesystem abstraction layer for PHP. Now if you haven’t worked with a Filesystem abstraction layer before (like me), you may ask yourself:

Why on earth should I add even more overhead for something basic as the file system?

Let me explain why it will be extremely useful. Say we have to build a small app in PHP which should download files from an FTP server, process them in some way, and store the result on the servers filesystem. No doubt, this can be done with PHPs build-in functions without a problem, you might say. Sure, but don’t forget the Unit Tes… dang! See?

Phase 2: Skepticism and Curiosity

The solution for our problem once more is: Abstraction. Abstract the process from any file system. Just use the same set of operations on some sort of adapter, no matter where or how the files are stored, and let the Abstraction Layer do the rest. You guessed it already, this is where Flysystem comes into play! Let’s start with a simple file download:

<?php

namespace My\Cool\Project;

use League\Flysystem\FilesystemInterface;

class Downloader
{
    /**
     * @var FilesystemInterface
     */
    protected $remoteFileSystem;

    /**
     * @var FilesystemInterface
     */
    protected $localFileSystem;

    /**
     * @param FilesystemInterface $remoteFileSystem
     * @param FilesystemInterface $localFileSystem
     */
    public function __construct(
        FilesystemInterface $remoteFileSystem,
        FilesystemInterface $localFileSystem
    ) {
        $this->remoteFileSystem = $remoteFileSystem;
        $this->localFileSystem = $localFileSystem;
    }

    /**
     * Download files from the remote file system
     * @param string $fileName The filename
     * @return bool
     */
    public function download($fileName)
    {
        $fileDownloaded = false;
        
        // This is the actual download part!
        if (!$this->localFileSystem->has($fileName)) {
            $fileDownloaded = $this->localFileSystem->writeStream(
                $fileName,
                $this->remoteFileSystem->readStream($fileName)
            );
        }
        // Done!

        return $fileDownloaded;
    }
}

Is that it?? Aww yeah, that’s it. When it’s with Flysystem, we only need a few lines of code, because it’s so intense!

To make the code it more clear, I called the filesystems $localFileSystem and $remoteFileSystem, but of course both filesystems could be remote ones, e.g. FTP and Amazon S3. This means that by using Flysystem, you completely decouple your code from the filesystem, so you’re prepared when the Business suddenly decides to “implement some totally easy changes”.

Of course our example is not 100%ly done, since we still have to create the Filesystem adapters we pass upon instantiation. But that’s ok, we need to do the configuration somewhere anyway. Below function is part of another class, which makes use of our Downloader class:

public function downloadSomeFile()
{
    $ftpAdapter = new \League\Flysystem\Adapter\Ftp([
        'host' => 'ftp.server.com',
        'username' => '123',
        'password' => 'abc'
    ]);
    $remoteFileSystem = new \League\Flysystem\Filesystem($ftpAdapter);

    $localAdapter = new \League\Flysystem\Adapter\Local('/tmp');
    $localFileSystem = new \League\Flysystem\Filesystem($localAdapter);

    $downloader = new \My\Cool\Project\Downloader(
        $remoteFileSystem, 
        $localFileSystem
    );
    
    $downloader->download('somefile.txt');
}

This is what I also like about using Flysystem: within the Downloader class we don’t have to deal with any path settings or such. It is completely done within the code which instantiates the Downloader, i.e. in the domain where it belongs.

it's so decoupled

Phase 3: Awesomeness and Why-didn’t-I-use-it-before-ness

I assume most of you didn’t even read until here, since you’ve been totally blown away and immediately started refactoring your code, giggling slightly insane. But wait, we’re not even fully through yet. Remember the question about Unit Tests? With Flysystem, it’s a breeze:

public function testCanDownloadFileFromRemoteFileSystem()
{
    // This mocks the remote server (could e.g. be an FTP) where we want to 
    // download the file from
    $remoteAdapter = new \League\Flysystem\Vfs\VfsAdapter(new Vfs);
    $remoteFileSystem = new \League\Flysystem\Filesystem($remoteAdapter);

    // Put a file on the virtual remote filesystem
    $remoteFileSystem->put('somefile.txt', 'just some content, does not matter');

    // Virtualize our local file system also, so no need to clean up or anything, 
    // since by using vfsStream, we don't even have to take care where to store 
    // the files during our tests.
    $localAdapter = new \League\Flysystem\Vfs\VfsAdapter(new Vfs);
    $localFileSystem = new \League\Flysystem\Filesystem($localAdapter);

    $downloader = new \My\Cool\Project\Downloader(
        $remoteFileSystem, 
        $localFileSystem
    );

    $this->assertTrue($downloader->download('somefile.txt'));
    $this->assertTrue($localFileSystem->has('somefile.txt'));
}

Flysystem even provides an adapter for vfsStream, a stream wrapper for PHP which helps greatly when you have to test filesystem-related functionality.

In simple terms, it simulates a file system in memory, which many (but not all) PHP functions can use.

Check it out here if you don’t know it yet: vfsStream on Github

So here we are: we just wrote a test for the FTP download in 5 minutes – without having to change a single line in the Downloader, without relying on a real FTP server. We could exchange the remote filesystem with Dropbox or even your Grandma’s closet within two minutes. Awesome. Now get a coffee and head over to your co-worker, who’s still struggling with the Downloader class and PHPs lovely native FTP support.

npm & Windows

Again, Windows can cause problems you would not have expected. In npm versions prior to 3, npm stores dependencies in deeply nested folder structures, causing errors on running ‘npm install’ such as

npm ERR! Error: EPERM: operation not permitted,
open '/var/www/my_longnamed_app/node_modules/
gulp-angular-templatecache/node_modules/
gulp-util/node_modules/dateformat/
node_modules/meow/node_modules/redent/
node_modules/indent-string/node_modules/
repeating/node_modules/is-finite/
package.json.29c584111ad29c009da7874ac2d19f55'

Obviously, this is a ridiculously long path and exceeds 260 characters – and Windows can’t handle that. Not sure why “they” (you know … MS…) came up with that idea, and why this is still not fixed in a modern OS like Windows 7 (I did not try versions above that). I guess it is just the pure essence of evil, the same diabolic power which brought the Internet Explorer amongst us. Let them burn, let them suffer, and let even pay money for it!

Even when you run npm on Linux in a VM on Windows with a Shared folder, you will still suffer from that. Oh, you beasts from the abyss, how … ok I’ll stop.

Luckily, npm3 supports flat dependencies, i.e. all dependencies are written into the top level of the node_modules folder. So if you run into the above problem, try

npm --version

If this still says 2.x, consider updating to npm3:

npm install -g npm

Delete your node_modules folder (not sure if this is really needed, but better go sure), then run

npm install

and all dependencies get installed without errors. Yay!

If you dare, you can have a look into the node_modules folder now. You’ll find loads of folders – the beauty of flat dependencies.