(or how I stopped blindly including shit in my composer.json)
I have absolutely no doubt this post will be largely disagreed upon by many in the PHP community, but I’ve had a terrible day and I’m hoping that the process of just getting this off my chest will be therapeutic in some way.
It started with a hydrator…
At work here at AdSpruce I’ve finally managed to start coding some, at least for part of our week. Our JavaScript SDK has scaled beyond our wildest dreams, and we hit a daily high this week with just under 10m SDKs delivered. Obviously, this raises problems of scale, and we’ve finally cleared a decent chunk of time to re-write and refactor our SDK with scalability, maintainability and responsiveness in mind. This should go down as a big win.
So, today I sat down and started writing the tests for our new lightweight SDK that offsets much of the work needed in the delivery of the adverts to workers via a Beanstalk queue. It should have been so easy. Things went well for the early part until I realised that I wanted to be able to extract and serialise our Device object to put it into the queue, and then hydrate it back into a Device object inside the worker. “No problem”, I thought. I’ve used Zend Framework 2 a lot (and as an aside, I’m now a ZFCA – woo hoo!), Zend\StdLib includes a set of hydrators that are specifically designed to cleanly solve this problem.
A quick look on packagist tells me that this component has no dependencies on any other component inside or outside of ZF2. That’s a win for reusable code, so I added it to my composer.json.
But wait!
Running my tests gave an unexpected error: 
It appears that Zend\StdLib\Hydrator\ClassMethods (at least) has an undeclared dependency on Zend\Filter. A quick conversation in IRC later and I find it also has an undeclared dependency on Zend\ServiceManager.
At this point I feel I should apologise to Marco – having a very bad morning is no excuse for being a fuckwit in IRC and rage-quitting when your opinions are questioned.
I also want to make it clear that my beef here isn’t with the undeclared dependencies in this component, that can be easily fixed and then the choice on whether to use it or not can be an informed one. My concern is over the number of people who would willingly and happily add the extra dependencies to their project in order to use this component. Zend\ServiceManager is currently 2,511 lines of code, and Zend\Filter is 7,351 lines of code. As far as I can tell (and don’t quote me here please, its a token guess based on 10 seconds of looking at code), the hydrator only needs the Word filters which are around 100 lines of code. So, using my amazing rounding techniques to make something look simpler that it is, by my maths, Zend\StdLib\Hydrator\ClassMethods imports 10,000 lines of code to make use of 100 lines. That’s 1% of the lines of code imported that are actually needed.
Yeah Yeah
So, this is where the contention happens. I can almost feel the overwhelming weight of apathy barrelling towards me as the literally tens of people reading this blog shrug in indifference. And that’s fine.
“Disk space isn’t a problem any more”
“So what?”
“You are a stupid idiot”
These are just some of the responses that I’ve got when moaning about this on IRC and on Twitter, and that’s perfectly fine. If you’re happy to let Composer pull in many and unknown dependencies so that you can use one or two classes out of the hundreds it installs, that’s completely fine. It’s your project after all. But I’m actually starting to sway away from this unchecked use of Composer.
Simplify All The Things
Around a year ago I first came across Ed Finkler‘s MicroPHP Manifesto. Of course, being a big advocate of the Zend Framework I immediately discounted it. But over the intervening months I’ve started to move further and further away from the full stack framework in some cases.
Of course, for some cases full stack is still my prefered framework of choice. Our client and admin back-end still leverages the power of ZF2; for one it makes user entry with Zend\Form (among other components) much, much easier. But when we rebuilt version 1.5 of our SDK, and now, with the forthcoming version 2 I want some of our sites to be as lean and lightweight as possible. This is where I am starting to really buy into the less-is-more ideals laid out so succinctly in the MicroPHP Manifesto.
I don’t want to have to maintain several packages for the sake of a small amount of code that I actually use in my project. I don’t want to have to worry about what versions of packages I’m running on my server or what versions of components are being pulled in as dependencies of my dependencies. And I don’t feel that this is a trivial as some others do. I don’t have the time to keep up with the changelog of all the components I have blindly installed to check if there are security fixes and I should update.
Composer Isn’t The Problem
Some people on Twitter saw my rant and wrongly interpreted it as a rant against Composer. It’s not. Composer is an awesome tool that in my mind has been responsible for a renaissance in PHP in the last 18 months. Composer has been the tool that has allowed me to blindly pull in multiple dependencies, but it’s not Composers fault. It’s my fault and the fault of the all the developers who have encouraged a community of thoughtlessly including code in projects.
So there it is.
The rambling rant of an annoyed man. In future, I’ll be thinking long and hard before including any dependencies in my composer.json, and I think you should to.
FIN
Gary Hockin @GeeH 
Very well said. Just wish I could have listened to you say it in your voice and accent. 😛 Do I see a new conference talk brewing?
I’m not sure about that Adam, but it’s a definite possibility, Funkatron has the monopoly on this topic though!
I agree! (And here is where the self-promotion begins. 😉 The Aura project is centered around the ideas you talk about here; it’s composed (heh) of fully decoupled, completely independent packages. There are no dependencies between any of the libraries. There are additionally project packages that bring together some of the libraries into a cohesive framework, but it’s done in such a way that you can remove any individual package and replace it.
http://auraphp.com
I have experienced the same issues, many of the components exists and mentions no dependencies and when getting installed it needs more and more. Hard time understanding it . And yes aura is awesome for it has no tight dependencies or hard dependencies and we have some awesome contributors . Have a look at http://auraphp.com/contributors/
Thanks for the comments, let’s not turn this post into an Aura love in though 😀
I had to re-read carefully your blog post to understand really what the problem is. It seems it’s just this:
> And I don’t feel that this is a trivial as some others do. I don’t have the time to keep up with the changelog of all the components I have blindly installed to check if there are security fixes and I should update.
Run composer update, and you’re done (in most cases). No need to follow changelogs, except for major versions of your direct dependencies. So yes, my main remark is: composer will install many sub-dependencies: so what?
I globally disagree with you, I am maintaining some open source projects and being able to rely on well-tested and powerful sub-libraries is extremely interesting, even essential. I know most of my projects couldn’t exist without all the dependencies: I don’t have the time, and sometimes the competence, to write those sub-modules. I believe many open source projects wouldn’t be as good as they are without those many dependencies. I even believe some wouldn’t even exist.
Also, sometimes what you see as 100 useless classes that could be done in 1 is because that allows for very good, strong, and tested code. And when you think you only use 1% of the code, sometimes you don’t imagine that actually it’s more like 50%. You could say “what’s the need for this useless layer of abstraction”, but the guys that spent months working on this really thought this through and built very good code, code that you might spend weeks re-write yourself (much less tested and stable).
Of course, I’m actually happily surprised given the exposure this post has got that you are the first one who disagrees with me.
I expected more if I’m honest.
It all comes down to if you’re happy to manage the dependency tree of all the installed components or not. Of course, it’s trivial to run the command “composer update” but you should never do this on production, you should run it on your development/staging environment, run your integration tests and then run “composer install” on production after deploying composer.lock. That isn’t trivial to me, particularly if there are breaking changes in a package you didn’t even know or care you used.
So yes, you’re right, running “composer update” is easy, but knowing when you _need_ to run it isn’t.
With regards to using other libraries, you miss my point entirely. I am absolutely not saying don’t use other people’s code, I’m saying think before adding things to your dependency list.
> running “composer update” is easy, but knowing when you _need_ to run it isn’t.
I usually run composer update regularly in the development branch, which then gets tested and deployed through CI. I don’t run it in soon-to-be-released branches. I don’t find it too hard.
I agree, this is very easy when you’re actively developing a product. What happens when the product is in production and you’re not developing it? How do you know that one of your dependencies has a security fix and you need to run “composer update” then? Unless you have eyes on all the release notes for all your dependencies then knowing you need to run “composer update” can be hard.
Yes but what are you comparing it to:
– A: you use a maintained library, and yes you need to update it to have security and bug fixes
– B: you develop your own solution which will never get fixes by itself
I can’t reply any further – but there is a C
– C: You choose your dependencies carefully and use the component that has no dependencies instead of 4 dependencies so you only have to manage 1 dependency.
The point of this post which you constantly fail to understand is not that you shouldn’t use 3rd party components, but you should think before including them.
“In future, I’ll be thinking long and hard before including any dependencies in my composer.json, and I think you should to.”
does not read
“In future, I won’t be including any dependencies in my composer.json, and I don’t think you should either.”
Nice of you to turn this discussion into something aggressive. Bye.
That’s not aggressive. I’m sad if you feel that it is.
Regarding security patches. I believe you have to worry only a about top-level dependencies, those in their turn, worry about their dependencies and deeper and deeper.
I understand your concern feeling like it you lose control over your project dependencies after some inactivity time. My solution is to cover your own code as much as possible (ideally 100%) and as soon as you update project dependencies – you run tests and make sure everything is ok. And, BTW, if something is out of whack, it’s actually a good sign you write your tests not in the best possible way. But that’s fine – next time you know!
@dV While in an ideal world you can only worry about the security of top level packages, it’s not realistic – especially in a package-happy place like PHP has become. Projects just don’t evaluate the components that make them up for security issues. A “not my problem” approach is a bad thing to follow, especially if you’re wanting to remain as secure as possible.
So, yes – keeping track of READMEs and CHANGELOG files is a must for right now and you definitely have to understand the risks of the packages you’re using, even in they’re not top level code. I *really* wish there was a better way to track this sort of thing (Symfony tried at least https://security.sensiolabs.org) but with all of the packages out there and no good audit process, it’s just not realistic.
Another thing to remember here – there’s also security implications involved in blindly pulling in new libraries. Especially in PHP, there’s not a tradition for reviewing libraries for potential issues. Even the OWASP list was recently changed to add a new item: A9 – “Using Components with Known Vulnerabilities” https://www.owasp.org/index.php/Top_10_2013-A9-Using_Components_with_Known_Vulnerabilities
Having Composer and Packagist is a nice, easy way to included these libraries, but as a reminder to those out there reading this post *please review that code* before putting it into production. Even better, pass it through a static code scanner to find the low hanging potential issues.
Personally, I see where you are coming from but have mixed feelings about it myself. I split my time between being a freelancer and being employed by an organization with one PHP developer (me!). My life’s goal — necessary so I don’t work myself to death — is to write as little code as possible and still get paid…and Composer et.al. have been a god-send in this regard. When I need to do something new I pop over to Packagist, search for a package that does what I need, give the code good once-over for any obvious design or security issues, pull it into my project, wire it up and hey-presto my job is done. They don’t change the underlying fact that any external code you consume must be thoroughly vetted before you deploy it, they just make the process of finding and using that code near-infinitely easier. As always, you have to be diligent about what you’re pulling in (directly and indirectly) and whether the trade-off is worth it.
Anyhow, that’s not why I came here…this is: I encountered the same issue today that triggered your Twitter and blogpost rant. The fault is that `ClassMethods` hydrator by default uses `NamingStrategy\UnderscoreNamingStrategy`, which is the source of the hidden Filter and `ServiceManager` dependencies. If you’re not particular about the array key format of your serialized data, pass FALSE to the `ClassMethods` constructor and you can use it without those additional packages installed.