Symfony 4 – REST API Unit Testing with Doctrine and Sqlite DB

There are plenty of benefits of using serverless architecture utilising 3rd party service. Lambda + API Gateway combo are a common one. I’d argue it is the way to go forward in most situations: https://martinfowler.com/articles/serverless.html.

Despite that, I think there are still few insights that we can gain by learning traditional web architecture for an API.


Architecture

The meat of REST API are basically just doing simple CRUD operations. Typically, we have few layers which are very similar between web frameworks:

  • Routing, which is where the list of routes are defined
  • Controller, which is where request validation is performed, data are retrieved from source database, data being processed and finally an HTTP response with some data is being returned

In bigger applications, we usually want split those huge controller responsibilities into a few more layers for long term maintainability:

  • Validation, which is where the validation of your request query parameters and payload content is performed
  • Repository, which is where queries or ORM calls being made to the source database
  • Data, which is where raw data being transformed to models for easier manipulation
  • Model, which is the representation of the data in the source database

Testing those layers separately can be tedious and are repetitive. I’d argue in most cases are unnecessary as most of them are to perform similar CRUD operations on every routes. In this post, I would like to show how we can perform end to end test of those layers, thus reducing the need of more granular testing strategies.

Config, Routes and Controllers

For simplicity in our example, we would stick all the logic in the Controller layer using Symfony 4 with routing annotation. Let say, we have a Product API that does few simple CRUD operations under MySQL and at the end we would like to perform and end to end test on all the routes.

config/packages/doctrine.yaml

doctrine:
    dbal:
        # configure these for your database server
        driver: 'pdo_mysql'
        server_version: '5.7'
        charset: utf8mb4
        default_table_options:
            charset: utf8mb4
            collate: utf8mb4_unicode_ci

        url: '%env(resolve:DATABASE_URL)%'
    orm:
        auto_generate_proxy_classes: true
        naming_strategy: doctrine.orm.naming_strategy.underscore
        auto_mapping: true
        mappings:
            App:
                is_bundle: false
                type: annotation
                dir: '%kernel.project_dir%/src/Entity'
                prefix: 'App\Entity'
                alias: App

src/Entity/Product.php

<?php

namespace App\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Table(name="products")
 * @ORM\Entity
 */
class Product
{
    /**
     * @ORM\Column(name="id", type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="IDENTITY")
     */
    private $id;

    /**
     * @ORM\Column(name="name", type="string")
     */
    private $name;

    /**
     * @return mixed
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * @return mixed
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * @param mixed $name
     */
    public function setName($name): void
    {
        $this->name = $name;
    }
}

src/Controller/ProductController.php

<?php

namespace App\Controller;

use App\Entity\Product;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

/**
 * @Route("/products")
 */
class ProductController extends AbstractController
{
    /**
     * @Route("/", methods={"GET"})
     */
    public function listAction()
    {
        $products = $this->getDoctrine()->getRepository(Product::class)->findAll();
        $results = [];
        foreach ($products as $product) {
            /** @var Product $product */
            $results[] = [
                'id' => $product->getId(),
                'name' => $product->getName()
            ];
        }
        return $this->json($results);
    }

    /**
     * @Route("/{id}", methods={"GET"})
     */
    public function singleAction($id)
    {
        $product = $this->getDoctrine()->getRepository(Product::class)->find($id);
        if (!$product) {
            return $this->json([], Response::HTTP_NOT_FOUND);
        }
        return $this->json([
            'id' => $product->getId(),
            'name' => $product->getName()
        ]);
    }

    /**
     * @Route("/", methods={"POST"})
     */
    public function createAction(Request $request)
    {
        $data = json_decode($request->getContent(), true);

        $product = new Product();
        $product->setName($data['name']);

        $em = $this->getDoctrine()->getManager();
        $em->persist($product);
        $em->flush();

        return $this->json([
            'id' => $product->getId(),
            'name' => $product->getName()
        ], Response::HTTP_CREATED);
    }

    /**
     * @Route("/{id}", methods={"DELETE"})
     */
    public function deleteAction($id)
    {
        $product = $this->getDoctrine()->getRepository(Product::class)->find($id);

        $em = $this->getDoctrine()->getManager();
        $em->remove($product);
        $em->flush();

        return $this->json([], Response::HTTP_NO_CONTENT);
    }
}

End to End Testing

The essence of end to end unit testing is to perform these few basic steps on every single test case:

  1. Setup an empty data source environment
  2. Populate with initial data
  3. Perform HTTP request, ie. CREATE /products with payload data
  4. Validate the HTTP response code and JSON response returned are correct
  5. Validate the data source now has the correct data, ie. upon DELETE request, the actual table row is actually removed

Ideally, this would an independent unit test which we can run in our CI/CD instance and hence does not rely on external application (ie. database server) call to be performed.

This is where an in-memory SQLite + Doctrine combo comes in handy. Utilising an ORM means we are free from writing SQL specifics vendor, thus we basically can be confident that this testing approach would also work for our actual database vendor.

config/packages/test/doctrine.yaml

doctrine:
  dbal:
    driver: 'pdo_sqlite'
    url: 'sqlite:///%kernel.project_dir%/var/test.db3'
    memory:  true

src/DataFixtures/ProductFixtures.php

<?php

namespace App\DataFixtures;

use App\Entity\Product;
use Doctrine\Bundle\FixturesBundle\Fixture;
use Doctrine\Common\Persistence\ObjectManager;

class ProductFixtures extends Fixture
{
    public function load(ObjectManager $manager)
    {
        foreach (['BMW', 'Mercedes', 'Tesla'] as $name) {
            $product = new Product();
            $product->setName($name);
            $manager->persist($product);
        }
        $manager->flush();
    }
}

tests/AbstractControllerTest.php

<?php

namespace App\Tests;

use Doctrine\Common\DataFixtures\Executor\ORMExecutor;
use Doctrine\Common\DataFixtures\Loader;
use Doctrine\Common\DataFixtures\Purger\ORMPurger;
use Doctrine\ORM\EntityManager;
use Doctrine\ORM\Tools\SchemaTool;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\BrowserKit\Client;

class AbstractControllerTest extends WebTestCase
{
    /** @var EntityManager $manager */
    private $manager;
    /** @var ORMExecutor $executor */
    private $executor;
    /** @var Client $client */
    protected $client;

    public function setUp()
    {
        $this->client = static::createClient();

        // Configure variables
        $this->manager = self::$kernel->getContainer()->get('doctrine.orm.entity_manager');
        $this->executor = new ORMExecutor($this->manager, new ORMPurger());

        // Run the schema update tool using our entity metadata
        $schemaTool = new SchemaTool($this->manager);
        $schemaTool->updateSchema($this->manager->getMetadataFactory()->getAllMetadata());
    }

    protected function loadFixture($fixture)
    {
        $loader = new Loader();
        $fixtures = is_array($fixture) ? $fixture : [$fixture];
        foreach ($fixtures as $item) {
            $loader->addFixture($item);
        }
        $this->executor->execute($loader->getFixtures());
    }

    public function tearDown()
    {
        (new SchemaTool($this->manager))->dropDatabase();
    }
}

tests/ProductControllerTest.php

<?php

namespace App\Tests;

use App\DataFixtures\ProductFixtures;
use App\Entity\Product;
use Doctrine\ORM\EntityManager;
use Symfony\Component\HttpFoundation\Response;

class ProductControllerTest extends AbstractControllerTest
{
    public function testListProducts()
    {
        $this->loadFixture(new ProductFixtures());
        $this->client->request('GET', '/products/');

        $response = $this->client->getResponse();
        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());
        $this->assertEquals($response->getContent(), json_encode([
            ['id' => 1, 'name' => 'BMW'],
            ['id' => 2, 'name' => 'Mercedes'],
            ['id' => 3, 'name' => 'Tesla'],
        ]));
    }

    public function testSingleProduct()
    {
        $this->loadFixture(new ProductFixtures());
        $this->client->request('GET', '/products/1');

        $response = $this->client->getResponse();
        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());
        $this->assertEquals($response->getContent(), json_encode(
            ['id' => 1, 'name' => 'BMW']
        ));
    }

    public function testSingleProductNotFound()
    {
        $this->client->request('GET', '/products/1');

        $response = $this->client->getResponse();
        $this->assertEquals(Response::HTTP_NOT_FOUND, $response->getStatusCode());
    }

    public function testCreateProduct()
    {
        $this->loadFixture(new ProductFixtures());
        $productName = 'Jaguar';
        $this->client->request('POST', '/products/', [], [], [], json_encode([
            'name' => $productName
        ]));
        $response = $this->client->getResponse();
        $this->assertEquals(Response::HTTP_CREATED, $response->getStatusCode());

        /** @var EntityManager $em */
        $em = self::$kernel->getContainer()->get('doctrine.orm.entity_manager');
        /** @var Product $product */
        $product = $em->getRepository(Product::class)->find(4);
        $this->assertEquals($productName, $product->getName());
    }

    public function testDeleteProduct()
    {
        $this->loadFixture(new ProductFixtures());
        $this->client->request('DELETE', '/products/1');

        $response = $this->client->getResponse();
        $this->assertEquals(Response::HTTP_NO_CONTENT, $response->getStatusCode());
        $this->assertEmpty($response->getContent());

        /** @var EntityManager $em */
        $em = self::$kernel->getContainer()->get('doctrine.orm.entity_manager');
        /** @var Product $product */
        $products = $em->getRepository(Product::class)->findAll();
        $this->assertCount(2, $products);
    }
}

Conclusion

Voila! There we have an end to end unit testing that we can run within our CI / CD platform and does not rely on a database server. This does not replace the need of full integration testing but it is significantly faster and easier to be run on regular basis and to test more complex logic. This is very important as it is considered good practice to adopt TDD approach in engineering teams.

Full source code can be viewed here: https://github.com/stellalie/api-unit-testing

Comments and feedback are appreciated 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *