Symfony CMF 2.0.0 First Release Candidate(3 years, 2 months ago)

We are getting ready with the CMF 2.0 release! We are in the process of tagging release candidates and hope to gather feedback in the next weeks, before tagging stable versions. The most important feature is support for Symfony 3. We've also created a #symfony_cmf channel on the Symfony Slack to improve collaboration and user support. Move in there, report an error or give us a high five! What is new? The upcoming versions will support Symfony 3.0 and higher, as well as the 2.8 LTS version. They require PHP 5.6 or 7. Deprecated code from the 1.x versions has been removed. If you use the Sonata Admin in your project, note that we moved the admins out of the individual bundles into the CmfSonataPhpcrAdminIntegrationBundle. MenuBundle will be released as 2.1, because there already was a 2.0. Note that the sonata admin classes for the menu bundle were also moved to the separate admin integration bundle. Then, some of the bundles also have new features: RoutingAuto supports multiple routes per document, e.g. to create REST APIs. TreeBrowserBundle got completely rebuilt, providing a much nicer tree with extensibility and better JavaScript libraries. It now also provides a form type for the tree widget. SeoBundle supports guessing the modified date and depth from the content. Alongside this, three new packages have been released: Resource, ResourceBundle and ResourceRestBundle. These packages provide a PHPCR and Doctrine PHPCR ODM repository for Puli. This way, you can access your documents like they are files. As Puli is not yet released stable, the bundles use a very limited set of Puli classes included in the Resource component. These packages will contain compatibility breaks when Puli is stable. Some Packages no Longer Maintained To focus our resources, the CMF core team decided to stop maintaining some packages for the time being. While they all have their merit, we just don't have enough time to work on all that has been created over the last five years. Rather than having badly maintained packages confuse users, we want to be transparent. The affected bundles are: CreateBundle, MediaBundle, SearchBundle and SimpleCmsBundle. If you want to help co-maintaining any of those packages, tell us in a GitHub issue or in #symfony_cmf of Symfony slack. How to upgrade? A lot of deprecated code was removed. Before upgrading, run your tests and check for deprecation warnings in the test log. Each repository contains a file that should explain the breaking changes when migrating to version 2. The instructions can be incomplete at this point. If you have time to test the migration, please reach out to us on GitHub or Slack if you find inaccuracies or information is missing. Documentation, Documentation, Documentation! Quite a bit of the changes are not yet reflected in the documentation. This will prove the largest part of the remaining work. You can become a new CMF hero by helping us with this task. Just a quick proof read of open pull requests or the documentation will already be very great!

Doctrine2 QueryBuilder Executable SQL Without Running The Query(3 years, 2 months ago)

On one of our projects that I am working on I had the following problem: I needed to create an aggregate temporary table in the database from a few different queries while still using Doctrine2. I needed to aggregate the results in the database rather than memory as the result set could be very large […]

Managing Background Processes within Symfony(3 years, 4 months ago)

When a web application reaches a sufficiently large size, it can become infeasible to perform all actions required within a single web request/response life-cycle. You may find yourself wishing to for example - batch up and send queued emails at particular intervals, or process payments asynchronous to the point in-time the user made the initial request. In this post I would like to discuss our changing use of background processes (both time-dependent and continuous) due to increasing throughput demands. Using Cron At MyBuilder we have grown to rely heavily on background processes, helping manage the day-to-day activities of the business. For time-dependent tasks (i.e. run a specific command every 30 minutes) we use the Cron job scheduler, to declare and execute the processes at desired intervals. We found ourselves using this so frequently in-fact that we created Cronos and the Cronos Symfony Bundle, which allows us to decorate Commands we wish to run using a PHP @Cron annotation. Using this system allows us to ‘dump’ the new crontab declaration upon each deployment cycle. This ensures that our background processes remain in-sync with the current released build. Upon reflection, this also proved to be a success in-regard to code-base understandability. It has been found that keeping the interval definition close to the code, makes it easier to reason about when entering a piece of functionality. Below is an example use-case of how we would use the @Cron annotation within a trivial Symfony command. /** * @Cron(minute="/30") */ class SendQueuedEmailsCommand extends Command { public function execute() { foreach ($this->fetchQueuedEmails() as $email) { $this->send($email); } } } Above we are specifying that we would like to send out any queued emails every 30 minute interval. The Need for More However, over-time the demand on these commands and throughput required has grown and grown. Reducing the invocation times gave us short-term aid, but eventually we were in need of always running processes. We even found that in some cases the need was so great that a single instance would not suffice, and having multiple workers running would be a requirement too. With this need to address we created the concept of a RunManager, which allowed us to provide Command’s with a looping process construct. Looking at the problem of sending queued emails again as a concrete example. Lets say that to mange the load at which we now enqueue emails to be sent, would require us to have an always running instance of the sender command. Below we have made some amendments, which will now iterativity keep processing queued emails that are present. class SendQueuedEmailsCommand extends Command { public function execute() { $manager = new RunManager(); while ($manager->isOk()) { if ($email = $this->fetchSingleQueuedEmail()) { $this->send($email); } } } } Run Forever In an ideal scenario we would kick off a process and it would run ‘forever’. Unfortunately certain events may occur throughout its life where-by the cheapest way of managing its state is to restart it (i.e. allocated memory exceeded). We got around this problem by firstly declaring (using the RunManager) how many of a process we wished to be present at a time. We were then able to employ our initial Cron infrastructure, to attempt to start the process at a given interval. This meant that if a process did die, the next Cron invocation would spawn a new instance. Guarding the command with a maximum process count, meant that we could be sure that only a certain amount of instances would be running at a given time. /** * @Cron(minute="/1") */ class SendQueuedEmailsCommand extends Command { public function execute() { $manager = new RunManager(['maxDuration' => 30, 'maxProcesses' => 2]); while ($manager->isOk()) { if ($email = $this->fetchSingleQueuedEmail()) { $this->send($email); } } } } Although this solved the problem of spawning a capped number of new processes, it did introduce a couple of issues. One of which was when you desired to have multiple of a given process running to accommodate for a particular expected load. From a cold-start you would have to wait for multiple invocation attempts from Cron to occur before you were at the desired processing power. Another point of contention was the unnecessary work Cron would do every interval in an attempt to create a new ‘potential’ process. More often than not the processes created by the Cron invocation would be ended immediately due to the maximum process cap. It seemed that although possible to use a time-based approach to represent ‘almost’ continuous processes, we needed to take a step back and review our thinking. Enter Supervisor As the story above highlights; there was a gradual transition from the need to always have a time-based approach (run every 30 minutes) to sometimes warranting a continuous approach (have two processes running at all times). As the demands increased we slowly innovated on our existing solution to cater for the throughput, however, what we needed to do was change our way of thinking. We needed to explicitly separate the concept of a time-based task which fitted well within the Cron philosophy and a continuous task. Changing our Thinking After some research we decided that Supervisor would be a worthwhile investment to incorporate into our stack. Supervisor is a process control system written in Python, which provides the infrastructure to achieve the exact goals we desired from a continuous process. Spawning processes as a child of the main Supervisor process, it allows the system to manage how many of a given process are currently running and the health of each one. When a child process dies for whatever reason the SIGCHLD signal is sent to Supervisor, which is then able to decide what action to perform (i.e. restart the process). Being able to now change our way of thinking from ‘when do I need to attempt to begin running a process’ to ‘I want this many processes running at all times’ was a hugely beneficial gain. Following experimentation it was time to try and incorporate this approach into our system, and transition over our ‘almost’ continuous processes, into true continuous processes. The Transition Transitioning from one system to another can be a challenging task, especially when it is so critical to the business. With this in mind, we felt it would be ideal not to have to change much in-way of the code relating to the existing continuous commands. As a result of this desire we created the Supervisor Symfony Bundle, which in a similar manner to its Cronos counterpart allows you to decorate Commands with a PHP @Supervisor annotation. These annotations are then parsed during the ‘dump’ phase and a Supervisor specific configuration is made available. Due to the similarities this meant that all that was required was to replace the @Cron with @Supervisor annotations (as shown in the example below). /** * @Supervisor(processes=2) */ class SendQueuedEmailsCommand extends Command { public function execute() { $manager = new RunManager(['maxDuration' => 30]); while ($manager->isOk()) { if ($email = $this->fetchSingleQueuedEmail()) { $this->send($email); } } } } The transition of these processes from Cron to Supervisor was successfully spaced out over a couple of days. As we had become accustom to automatically dumping out the crontab upon each deploy, we were only required to do the same for the Supervisor configuration. Lets be Safe There has been one key omission from this story so far, that I would now like to discuss. Another pain point within our Cron setup, was how to safely handle the transition from one version of a process to another. At this time the RunManager was only able to safely exit activity based on memory, processor and duration limits. If we were to attempt to terminate a process mid-way through a critical activity we would not know of the lasting side-effects. This meant that we would have to wait for all process duration limits to be exceeded before we could be sure that processes were stopped, and new versions could be started. Ideally, what we desired was to be able to safely halt execution of all current processes, run the deploy and then bring back up the processes again with any changes that had been applied. Signaling Upon further research, and with Supervisor’s signal based approach in-mind, we decided to take advantage of Unix signaling. We did this by providing another RunManager predicate to safely exit the looping construct found in these processes. Upon an update request, Supervisor sends out a specified signal (i.e. SIGTERM) to each of the running processes. This meant that we could use the pcntl PHP extension to listen out for these signals and react accordingly to safely stop the process. Below is an simplified implementation of how we incorporated Unix signaling into our RunManager. class RunManager { private $ok = true; public function __construct(array $options) { // ... pcntl_signal(SIGTERM, function () { $this->ok = false; }); } public function isOk() { pcntl_signal_dispatch(); return $this->ok; } } With this in-place we were now able to perform a Supervisor update upon each deployment. This would safely finish any work the process was doing (i.e. finish sending the current email) before shutting down and allowing a new version to be spawned. Unlike the duration-based limits we relied upon in the past, we could now be sure that a new deployment meant that only newly deployed processes were being run. SIGSTOP Upon review, I feel that this change has been a great addition to our stack and process. We are now able to clearly differentiate between a time-based command (of which there are many still present) and a continuous process. Although complexity of course has increased in-regard to the inclusion of Supervisor, using the right tool for the job has paid off greatly.

Episode 16 - Building a better bundle(3 years, 4 months ago)

In this episode we discuss what makes a good Symfony bundle, and how you as a bundle author can build a better bundle. Photo credit: bundle by foam (CC-BY-SA)

Sylius v1.0.0 Alpha released!(3 years, 5 months ago)


How to get more than Request in Controller Action(3 years, 6 months ago)

You already know you can get Request object in your controller action. Cool, but there is more. In Symfony 3.1 is new Action Argument Resolving feature, so you can get any service you need. With a bit of work. Today I will show you how. Disclaimer: What happened to controller constructor injection? I still prefer constructor injection in controllers. Are you asking why and how to achieve that in Symfony? In 3 minutes you will find out in my other article. I will wait here... ...ok. Now you are [...]

Episode 15 - Live from Barcelona(3 years, 7 months ago)

In another of our conference specials, Tobias, Magnus and Ryan is live from Symfony Catalunya 2016 in Barcelona. We talk about the conference, and, trying make the most of being around some of the experts in the community, we talk to Marc Morera and Michael Cullum about building local PHP communities. Photo credit: Jeremy Mikola

Symfony Security Roles vs. Voters(3 years, 7 months ago)

In my previous blog post I've explained the basics of authentication, authorization and how this is dealt with in Symfony. Due to the size of the post, I've left out several important topics such as roles and voters; Both an equally important part of authentication and authorization. A common misconception is that roles should be used to check permissions. In fact, they should definitely not be used to check permissions!

Sylius Roadmap Update(3 years, 7 months ago)


User switch with custom restrictions in Symfony(3 years, 8 months ago)

An example on how we added extra rules to the switch user functionality of the Symfony security component.