How to structure a Zend Framework application and it's dependencies

I have been thinking a bit recently how to manage dependencies and how to structure Zend Framework based applications to make the code less coupled, more testable and less dependent on the global scope.

I don't mean to be negative but I am not too happy about the web application structure that most articles and books present. In Zend Framework world controller seems to be the place when things get done. Controller is the workhorse and this is where all the logic seems to be buried. It also seems to me that model in MVC is reduced to database integration but there is no services layer for some reason. Where ever you look you will see the same examples with controller doing all the work and models being simple Zend_Db_Table or Zend_Db_Table_Row instances. You will not see business logic focused classes, Controller or DB Model, thats all you can choose from.

The problem I see with this solution is that controllers are not reusable. Controller action is coupled too tightly with way too many things to be reusable. You can not call controller action from a cron or a REST server. You can not easily reuse controller action from a queue processor. Why? Because it depends on session, request, cookies, response, views etc. Controller has way to many things on its mind, if you want to reuse it you would have to satisfy all its dependencies which get out of control easily.

What you can see in the wild is even worse. People use $_POST and pass it around to the view or models. In result everything depends on everything and you have a true spaghetti code.

So how to decouple components in Zend Framework applications?

The first thing that we should aim for is to reduce the amount of code that is coupled with Zend Framework itself and especially with the request scope. We should try to create decoupled services and POPO (Plain Old PHP Objects) which handle business logic. They should not be aware of session nor request/response/cookies etc. They should depend solely on what is provided in constructor and method arguments.

The less class depends on the better. If class depends on a cookie and DB that is already too much as it means that the class is tightly coupled with both the presentation and database. In multi layer architecture you should not allow lower layers to call or depend on higher layers. You should also not allow direct use of bottom layers by top layers etc. You spend more time on to achieve this separation but the benefits are huge as you end up with a system that is structured and maintenable.

After giving it some thought I came up with the following diagram of how dependencies and scopes could look like in a Zend Framework based application.

There is a lot of rules here represented mostly by the dashed arrows.

Service is king

In this utopia-like diagram services are where the job gets done. Services present a simple API to perform all the complex operations behind it. It can be anything, for example:

$userService->disable($user)
$captchaService->generate()
$purchaseService->completeOrder($order)
$cache->load($key)

The point is that services are kept away from the following:

  • request
  • response
  • session
  • cookies
  • view
  • current user context (beside permissions)
  • request parameter names (associative arrays containing POST are not cool)

This means that service class has a well defined API. It exposes simple methods with parameters that have structure (interfaces or classes not associative arrays).

It also means that services can depend on each other and on the external libraries and/or database. Here I would divide services to two categories. Database aware and database agnostic. The difference is important from testing and reuse point of view. If your class depends on tables and db structure you wont be able to reuse it or test in isolation easily. If your service depends only on the interfaces injected in the constructor then you can easily replace them with mocks or wire the application differently at runtime.

Last important thing about services is that they do not keep state. You should expect a service to do the job and after service method returns something service should be ready to take another request. Then it does not matter if you want to process one payment or a queue of million payments. Your payment service takes a payment object as argument and processes it. It can succeed or fail but it should still be ready to take another payment without aftereffects. It is also very important if you want to provide REST or SOAP API for your application. Having stateless services you can easily invoke them from your API layer or crons and you will not need any modifications.

Services are classes that perform operations based only on the service method arguments. If we create services this way they can be easily invoked from controllers, plugins, command line, queue or Soap entry points. They are also much easier to test as we know they only depend on the arguments (an constructor injected services/objects).

Database aware/agnostic services, why should we care?

On the diagram I separate them to highlight that dependencies on database are heavy. To make these services simpler we do not allow them to depend on 3rd party code directly or talk to external services. If a DB-aware service needs to generate a PDF it should use a PDF service. Then code of DB aware service stays simpler and PDF service takes responsibility of PDF generation (which is not trivial).

Please note that there is no dependency in the opposite direction. Paypal service facade does not call the database-aware service. It should not. If PayPal needs to expose endpoint for payment notifications it should be a controller calling a payment service which in turn would ask PayPal service to process the notification (extract reference id and status).

What can models do?

Starting from the bottom we see models talking to the database. They have no idea what is a request, they are not allowed to call services nor controllers. They can not use external libraries nor session, cookies etc. They should not be coupled to request parameters either. Models should be pretty dumb as they all extend Zend_Db_XX so reuse will be limited. I think it is ok to let models depend on other models and let them perform different queries.

How to incorporate 3rd party code?

In left bottom corner I placed external libraries as I think it is important to keep them in mind.

Every application uses some 3rd party code. If we want to keep our application safe from external changes we should separate it with a wrapper (facade) service. We never know what bugs are there and if we won't have to replace the implementation. We do not want to pollute our API with 3rd party standards and exceptions either.

The best way would be to wrap external libraries in form of simplified services. These services simply delegate to 3rd party code or perform operations to hide complexity. Interfaces that expose exactly and only what we need. If we want to resize images lets create a facade with simple API to resize images. Our application does not have to create any external instances, it is separated from implementation details and it is easier for us to change the behavior if necessary as its all in the image resizing service.

Services that encapsulate 3rd party code components should not be aware of request/response/session/database either. They should focus on being adapters between 3rd party code and our service interface.

How to integrate with external systems?

In the same layer of services we provide services that talk to external systems. Again they provide a function of adapters as they would not have any major business logic.

How to support crons, SOAP, command line tasks and Queues?

Now that we already have a services layer. It is very easy for us to add a queue processor that populates service arguments with queue data and executes a service method. We can add SAOP endpoint with methods that map SOAP arguments to our service method arguments and we should be ready to go. Crons are equally easy to implement. We can have complex tasks reused across web/soap/cron interfaces.

The VC related code (view and controller)

On the very top of the diagram we have a bunch of different components. We have Front Plugins, Controllers, Action Helpers, View Helpers, Forms, Views and Partial Views. It is quite a lot so lets break it down.

Front Plugins

I think front plugins can not escape coupling to request, cookies and session. Front plugins can be used for stuff like redirecting, geo location, permissions etc. In many cases these components will depend on request, session and cookies. Plugins can also be involved in preparing user context and permissions.

Front plugins do not really touch the views but their life cycle is controlled by MVC and is quite closely related to controllers so I put them in the same group of view related classes.

Controllers

There is no controversy about the role of controllers any more I guess. They are the bridge between user interface and services. Controllers use request, response, cookies and session to assemble service methods' arguments. Controllers do not contain real business logic. They do not talk to Zend_Db_XX classes, they do not use 3rd party code etc.

Controllers are using forms and populating view parameters. Controller actions can also use action helpers.

Views and partial views

I think this area is safe and simple too. Views should only access values populated by controllers and view helpers. View files should never access session, cookies, request parameters nor invoke service method. They render the user interface and this is their only responsibility.

Action helpers and View helpers

Now for the more controversial components, Action helpers and View helpers. I think action helpers are code that should be reused across controller actions but still depends on request / response / session / cookies. We should not have a lot of it but I guess in any application there will be components like this so better to encapsulate the similarity then copy paste.

I am not sure which components would fit best into action helpers. Should they be allowed to use forms? On the diagram I did not draw the line suggesting that action helpers can not use forms. I am also not sure yet should action helpers be allowed to set view parameters? I think they should not, as it would be difficult to keep track of which action helper sets which values and what has to be called to make sure that view has all the variables it needs.

View Helpers are the biggest controversy here so lets spend some more time on it.

View helpers, smart or dumb?

Looking at different frameworks you will find that view helpers are usually just simple HTML preprocessors. Usually they look very ugly having a lot of HTML generation inside of the helper method.

But where do we put standalone components in Zend Framework MVC? If I want to have a user login bar at the top of every page should I populate partial view variables in init() method of base controller? Do I have to call action helper or set view variables in every action?

What if I want to have a refer a friend widget on some of the pages? Do I have to add variables into my view in every action? Would it not be easier to use a view helper instead and treat it as a component? Would it not be nice to say “give me a featured product here” in the layout or some view and have the recommended product details loaded for you?

This issue is probably the only thing I am not really happy about when it comes to MVC frameworks like Zend Framework. There is no solution to my problem. I want a mini MVC component with its own life cycle. I want to be able to decide in the view should it be evaluated or not. View helpers fit quite well into this use case. I assemble service method arguments, call service, get response, populate partial view and render it. Job done, featured product is on the page.

Maybe it could be done on a layout level somehow? I am not sure if it has to be in the view helper but I it seems flexible.

I am not really sure if this is a best way to do it but I think view helpers should be allowed to do simple controller-like tasks and ask services for some data if necessary to render the widget. It may be a call to get user's flicker preferences to show the right images in the side bar or it can be a query to generate menu tree based on permissions and preferences. I believe that as long as the service is doing all the heavy work and helper just consumes it then we should be fine.

Purists will say it is an violation of MVC, but MVC is not a web pattern. It was designed for desktop applications and the Web twist on MVC is quite different from the original pattern. When you think of it and look at the diagrams you will see arrows from view to model. Zend calls models db tables and rows, our models are much more sophisticated, our models are services. So my gut feeling tells me that allowing view helpers to access service methods is fine.

User context

The last piece of the puzzle is the user context and translation which have to be available in all the view related code. We will make decisions in the view based on permissions (show or hide option). We will make decisions based on user profile, preferences etc all the time so having a well defined user context available in VC makes sense.

I believe services should not be aware of the user to keep them simple, if you want to incorporate permissions into service logic you will have to come up with an actor parameter that is passed around to every service call or make it available somehow via static scope. But then you make your services more coupled to the user and it may be harder to test them in isolation or use them from user-less scope like cron or message queue.

I think it would make sense to prepare user context in the front plugin or somewhere like this and decorate the request object to make user data available across VC layer. Your user object could have whatever is commonly used:

  • permissions
  • userId
  • personal details
  • preferences

I would not make the user object responsible for talking to services though. I think it should be a data container for VC layer to grab the basic information from.

Discussion time

Well the article got a bit longer than i intended but i hope i did not bore you to death :) Please let me know what do you think and share some thoughts with me. I am especially interested in your experiences with services and independent components.

Cheers

Comments

I was wondering how I would

I was wondering how I would go about setting up a db aware service in a ZF 1 modular application. For example, your "PurchaseService". So far I have a fair amount of abstraction, but I'm not sure how to approach the PurchaseService to make the project better.

2012-07-25 06:11
Lee Robert

Awesome Article!

Awesome Article!

2012-06-13 01:08
Anonymous

Thanks for the great article

Thanks for the great article ! Just wanted to add that as a far as I know it is perfectly OK for the view to pull data from the model. I think it's actually known as the MVC pull variant (as apposed to MVC push, where the controller pushes data to the views).
Now, maybe it's not a good idea to mix the push and pull models too much. But when you limit the use of the pull strategy to the view helpers and use push everywhere else, my feeling is you get a lot more flexibility and still end up with well maitainable code.

2012-02-12 14:06
Leven

This looks actually pretty

This looks actually pretty similar to what a rather large ZF application I work on (http://www.wantlet.com) uses. It wasn't actually originally designed like that, but rather evolved based on how much need there was to have certain functionality abstracted - in a rather "agile" sort of way.

The main difference in our case would be that since we use Doctrine 2, most of our services have a dependency directly to the EntityManager, but also the entities themselves contain much of their own logic. Despite this, the services are easily unit-testable thanks to all queries being abstracted into repositories, which are fetched through the EM (and easy to mock).

I'm actually planning on writing a bit about the architecture, and how it evolved to this point on my blog at some point.

2011-07-13 11:00

>> I think it is ok to let

>> I think it is ok to let models depend on other models and let them perform different queries.

Yes; however, be careful to inject these dependencies rather than hard code their instantiation into the model that needs that dependency. I'm pretty sure you are doing this; however, your article didn't explicitly point this out.

Nice article...thanks for sharing this.

2011-07-12 09:35

Hi Artur, it looks like you

Hi Artur, it looks like you and I share similar ideas on how to structure a ZF1 application. What you outlined above is roughly what I've been practicing for the last year or so. Some thoughts...

1. Models:

I think you're confusing data access with models. When I was using Zend_Db, my models were POPO and persisted via a data mapper. This allowed me to stick business logic in the models (where it should be), while keeping a clear separation from persistence details. Generally speaking, I put business logic that relates to a single entity in a model and more global/collection logic inside of a service.

A thin model falls into the realm of the Anemic Model: http://martinfowler.com/bliki/AnemicDomainModel.html

As the pullquote from Eric Evans mentions on that page, services are supposed to be kept thin, not contain the bulk of the business logic. I've seen fat services work, though and god knows I've been guilty of creating my share of them on occasion.

2. Action Helpers

These helpers are purposed specifically to reduce duplicating code. With that in mind, I think that it should be allowed to do anything that a controller is allowed to do. The difference between helpers and a base controller is that logic in a base controller could be run on every request, whereas action helpers are explicit. You know when you're using an action helper, so if it sets a view variable, or instantiates a form: that's fine.

3. View Helpers

I'm not sure who the "MVC purists" are, but in more formal explanations of the MVC pattern, you'll notice that the view is allowed to talk to the model layer. With that in mind, I'm perfectly fine with view helpers collecting the data they need without it being spoon-fed data via the controller.

In fact, its usually better to have the view helper get information from the model layer because it keeps the complexity down: You don't need to worry about, "Oh well... am I still using the view helper on this action?" Let the view helper worry about the data it needs, if you want to be able to pass custom data, just add a method to the helper that enables you to do so.

Just some thoughts. I think you're onto something good here.

2011-06-25 18:08

If you're using

If you're using Zend_Application to bootstrap the application then you immediately have this available in the controllers as a possible Context class. In Symfony 2 that might be similar to having access to $this->container. In ZF1 the Application class is available via the bootstrap arguments to the controller, so in the base controller class one might have

public function getContext()
{
return $this->getInvokeArg('bootstrap')->getApplication();
}

Now if you use the Context/Application to provide your Services, you can automatically set the current Context class into those services as needed.

Whether or not you might want to put the DIC into a separate container is up to you, however my having concrete methods for retrieving particular Services in the Application class does then provide code completion support and is a little cleaner in the end (rather than having a multitude of $this->container->get('Service').

Basically what I'm referring to is

http://www.agavi.org/apidocs/agavi/core/AgaviContext.html

Regarding chaining of ActionHelpers are you talking about creating a Web Flow (sequential pages)?. Although not extremely elegant, I ended up having one main ActionHelper that specified the list and order of execution for the other ActionHelpers.

Although it there isn't much documentation regarding the ZF1 Service objects, e.g their usage, I think it better to keep the naming convention and their location as per the ZF1 expectation (in the modules directory) and just rename (or reclassify the other Service objects - hence for me Manager seemed to work out in this regard).

What I meant about a service maintaining state, I meant about in memory for the lifetime of a request. Which a ZF1 module service component could be a good candidate for and this Service could then cache in memory the results from a Manager. Ultimately what is happening is just trying to assign different classifications to the different types (or behaviours) of Service objects.

At some point in the application code data is handed off to the DB components, if the only dependency of that DB Aware Service object is the Database, then it would be better to classify it as something else so as to distinguish this type of Service object from the rest, e.g a Data Mapper. Managers can access other Managers, however a DataMapper does not collaborate within any other component other than the database.

For example a modules/default/service/UserService object could be used to retrieve the current UserModel. The UserService caches the UserModel in memory (stores it in a local var) so that it doesn't have to re-query the session (etc). When initializing the UserModel for a logged in user you'll probably need to query the database, this is handed off to the UserManager to do - at which point it is up to the UserManager whether or not it is going to retrieve this information via Data Mapper or directly from the database (the amount of abstraction is refined as/when needed). When inserting or updating a UserModel, the UserModel is passed to the UserManager which will then handle the persistence to the database (probably via DataMapper).

In the end there are cleaner solutions by increasing the separation of concerns. However an initial pragmatic does allow scope for these refinements to occur as and when needed.

2011-06-14 07:06
Greg

Hi there Greg. 1.

Hi there Greg.

1. "Zend_Application and or clearer definition of DIC?": DIC is a whole new story. I am playing with symfony DIC right now and looking at ways to integrate nicely with our existing ZF1 code. I still believe it will work well but at this stage i am not sure how will it look like :) I agree that services should be able to use other services without need to satisfy their dependencies. If i need to store an order i dont want to know that orderService needs a logger, cache instance and whatever. I will look closer at zend application :) do you have any tutorial or link on hand that would show in more details how it can be utilized?

2. "Action Helper could also be used as reusable Controllers?": Yeah i was thinking of action helpers or chaining actions with forward or something. So that i could route execution through multiple action helpers or controller actions and each of them could set view etc. Difficulty i have with it is how do i easily control the flow? at some stage i will have to make a decision which action helpers / actions will have to be executed and in which order. Would it be in the end of my controller action? Would it not lead to a lot of duplication and complex flow? in a way it would be cool as components could be smarter. They could handle forms and do whatever. It is interesting idea and i will have a closer look at it.

4. "Where in the application code do Services reside?": I would put services in a separate folder structure. There will be a lot of code :)

5. "Some Services could maintain state?": Yes i agree. If this is the logic that is needed i don't mind. I would prefer to have a generic cache in front wired up via DIC or something like AOP but if service needs to be aware of caching its fine. Services could have some state but mainly configuration and instances of interfaces they depend on. Service would not have a session with the user though. So there would be no init(), doSomething(), getResult(), commit() or anything. If service wants to use cache behind the scenes it can (via a cache instance). If it aggregates some metrics on performance or tracking that is fine too. My point was that you should be able to use it as a remote service and that each service call is separated. Different clients can call service methods in different order from different sources and service should still work fine.

6. "Couldn't DB Aware Services be Data Mappers or Repositories?": not sure if i got the idea :)

7. "Are all Models directly tied to the Db Tables?": Well in my current project they are :) would be nice if they did not though :) It is a good point. I will have a look if there is any hope of changing that in our code.

8. "An alternative schematic for the User Context": Please explain a bit more how would that work. It is area that we struggle with at the moment and i want to make sure that i got your point. :)

Thanks a lot for a lengthy comment Greg, i appreciate it.

Cheers

2011-06-14 03:54
admin

Some questions or comments

Some questions or comments might be,

Zend_Application and or clearer definition of DIC?

I found that utilising the Zend_Application as a central Context class throughout the application smoothes out some of the development hurdles especially by being able to provide a DIC (Contextual Lookup). Either way this DIC enables Services to collaborate with each other with them having to know about the dependencies of the others.

Basically this Context class would provide a layer between the View Related box and the Services Layer - e.g Context or Container Aware.

Action Helper could also be used as reusable Controllers?

Although may be not elegant, it is a way for a Controller to pipe the request to a particular Action Helper, e.g. for a particular component there might be various types of sub-components and each could have their own ActionHelper (aka Controller).

Couldn't a mini MVC inside a View Helper be expensive?

Not sure that would be my first choice.

Where in the application code do Services reside?

In the a ZF1 services are located in modules directory? Wouldn't the Service classes be better located within their component class directory?

Some Services could maintain state?

What happens when you want to the cache the results of a Service call in memory? This is where I distinguish between Services (in the modules dir) and class Service components (Managers).

Couldn't DB Aware Services be Data Mappers or Repositories?

What are the dependencies of the DB Aware Services, if it just the database connection then it might be clearer labelling them more closely to their purpose?

Are all Models directly tied to the Db Tables?

In the current Quick Start guide for the ZF1 models no longer extend DB Table.

An alternative schematic for the User Context

UserService -- maintains state of the current UserObject
UserObject -- Is the user model (doesn't have to be tied to a DB Table)
UserManager -- does not maintain state is a proxy to UserDataMapper (kind of like Doctrine's Entity Manager)
UserDataMapper -- Primarily governs inserts, updates and deletes. Can also handle usual queries e.g. findByOne etc

In the alternative above, the UserManager could start out providing all the functionality that a Data Mapper would, but as the code (i.e. collaboration of services) becomes more involved it would be better to further abstract the User related DB operations to the Data Mapper.

Overall I think what you outline is a good way to go.

2011-06-13 19:59
Greg

Post new comment

Image CAPTCHA

About the author

Artur Ejsmont

Hi, my name is Artur Ejsmont,
welcome to my blog.

I am a passionate software engineer living in Sydney and working for Yahoo! Drop me a line or leave a comment.

Follow my RSS