Yes, I wrote earlier on how to have a lack of confusion on Heroku. But sometimes, things can still be a little confusing. I recently decided to play with Ruby 2, Rails 4, and a new app idea, all at the same time, via Heroku, launching Pull Request Roulette.
All went well in development mode, trying it out on my own box. Likewise in test mode, where I was trying to follow fairly strict BDD and TDD. In production mode, though, and only in production mode (where it was most embarrassing and troublesome, of course), the "Take" feature, whereby someone commits to review a Github "pull request", wasn't working. It was acting like I was trying to view the record of the pull request, which (at this stage) consists only of the URL. It's all the worse because there are only two basic features, Submit and Take.
Production mode on my own machine... showed the same symptoms. Aha!, I thought, at least it's not Heroku-specific. Tracking down Heroku problems can be a pain, due to the layer of indirection. The logs (more easily accessible on my own machine) pointed to an asset problem. Heroku precompiles on upload (unless disabled of course), so that wasn't it. What could it possibly be?
I Googled a gaggle, and it didn't bring me much giggles, especially after spending enough hours to develop late-night-goggles. Finally I stumbled across a post on StackOverflow that referenced Heroku's documentation about Getting Started with Rails 4.x on Heroku. I could have sworn that I had read that, and followed all its advice when setting up Pull Request Roulette initially... but the referenced Heroku's instructions to include some gems that didn't sound familiar.
Sure enough, when I opened up the Gemfile, they were nowhere to be seen. Adding them fixed the problem.
So now, if you want to have someone review a pull request you've put together for an open-source project on Github, someone can now commit to review it. For now, though, you won't know who it was, and there will be no record. Want to add that feature to the project? Fork it and create a pull request... and you can submit that one to Pull Request Roulette.
Saturday, April 27, 2013
Tuesday, April 9, 2013
Even LESS Heroku confusion!
Way back when, I wrote about easily using Heroku for both staging and production versions of your web site, without much confusion over which was the just plain "heroku" remote, by explicitly designating them as "staging" or "production" instead.
Now, to make it even simpler, I've written a tiny little script, that lets me just do "script/deploy staging" or "script/deploy production". Another advantage of using a script is that I can do additional mundane overhead tasks in there. In this case, I turn Heroku's maintenance mode on before pushing, and off afterward.
Now, to make it even simpler, I've written a tiny little script, that lets me just do "script/deploy staging" or "script/deploy production". Another advantage of using a script is that I can do additional mundane overhead tasks in there. In this case, I turn Heroku's maintenance mode on before pushing, and off afterward.
#! /bin/shTo adapt it to your own site, simply adjust the APP and, if needed, BRANCH vars. (If you want to get fancier and deploy the master branch to the production remote and the develop branch to the staging remote, that's a whole 'nother story... but should be pretty trivial to code up.)
APP=codosaurus
BRANCH=master
SITE=$1
if [ -z "$SITE" ] ; then
echo "Must specify site (staging, production, etc.)"
exit 1
fi
heroku maintenance:on --app $APP-$SITE
git push $SITE $BRANCH
heroku maintenance:off --app $APP-$SITE
Friday, March 29, 2013
All Right, Break It Up!
On a recent project, I worked on a very large Ruby on Rails application. My temporary employer joined the project after it was well underway. It was composed of four "portals", three of which were already mostly written, and fairly large and complex. Our work was mainly (and mine was exclusively) on the fourth, at the same time as the client was working on it too.
An app composed of four clearly defined parts could be written as four, or at least three, separate Rails engines mounted on one app. Or possibly as four separate apps with a library of shared code, all accessing the same database. Or if the four parts are relatively small, one application that includes them all. In this case... it was monolithic, but not of small parts at all. We repeatedly recommended breaking it up, but the client just didn't want to. Funny thing about consulting is, the client is always right -- not just even when he's wrong, but especially when he's clearly, provably, absolutely flat-out dead wrong.
"So what?", you might be wondering. "What good would breaking it up do, aside from pleasing the ivory-tower purists?"
The problem is, this has quite an effect on the rate of progress. Even if the engineering of the code itself is very clean, so you don't have god-objects causing churn and interference in the actual code (as the client did), and you have excellent developers making rapid progress in developing features (as, thankfully, both the client and we did, he said ever so humbly)... look at what happens when someone's just trying to commit a feature. The skill level of the people involved doesn't matter at all at this stage, it's just pure statistics.
Suppose you are on a team of about twenty developers, making a web app to run a school, with different portals for teachers, administrators, students, and parents. You and a colleague pair program on the next highest-priority ticket from the backlog in the issue tracker. It turns out to be in the parent-portal. You create a feature-branch, off the master branch. You get the feature passing the automated acceptance tests given in the user story, pull the latest master, merge it into your feature-branch (resolving any conflicts), and run the whole test suite. It passes, so you didn't break anything.
Now what? Merge into master? You try... and get rejected by the revision control system because someone committed changes to master while you were testing. Not surprising, as there are twenty of you, all hard at work, and maybe some of them aren't pairing so there are even more than ten features under active development... and the test suite takes an hour to run. The changes were in a different portal, so unlikely to interfere... but better safe than sorry. Nobody wants to be the jerk who broke the build, or even the Continuous Integration server. So you pull the latest master, merge it into your branch, and run the test suite again. Still green, so your changes don't break anything that was merged into master during your previous test run. Nice to know, but it means that one of the test runs was a total waste of time. Maybe you used it for something else productive... or not. Either way, it's nearly certain that your mental house of cards concerning that feature has utterly collapsed, because you've been thinking about something else.
So now, insert the above paragraph, again. Maybe you pull some fancy tricks like parallelization and extensive mocking and stubbing of slow services and expensive object creation and so on, and cut the time down to fifteen minutes. That's still plenty enough time for it to happen again. And again. Lather, rinse, repeat, ad nauseam, which gets to feeling like ad infinitum... especially to your client, who is breathing down your neck, waiting for this feature.
Nobody's happy, not him, not your boss, and certainly not you. You're a good developer, you want to be productive, you believe in testing... but you hate the inane futility of having to do it over and over in vain.
Now let's consider what it might be like if the app had been broken up into separate engines or apps. Even assuming a fifth item (an app to mount engines on, or a library of code shared among apps), the size of the codebase you need to deal with at one time is still cut down to 40% (assuming all pieces are of equal size). You could possibly make it 20%, but let's even assume the worst (within the assumptions already made). How would that help with this problem?
First, only 20 to 40% of the changes being worked on (assuming equal distribution of current work) are likely to block your code from being accepted by version control. (You're probably only working on one of the apps/engines, and maybe the library or main app but usually not.) Call it 30% to keep it simple. This means only 30% of the probability that any given change, to the overall system, will make you run the test suite again.
But wait! There's more! (Or rather, there's less!) The time it takes to run the test suite, should also get cut to about 40%. (Less when you don't change anything the shared code depends on (so you only have to run one portal/app's tests), same when you do (that plus the shared code's tests), more when you change the shared code (everything). Call it a wash to make the math easy. Still certainly a smallish fraction.)
Not only is that an instant time-savings right there, and not only does it make it easier for you to stay mentally on-task in case some more changes are needed, but:
Combine these two factors, and they mean you have only 12% of the original probability of an interfering change happening during your tests, making you run them again. Suppose the original probability was 75%, so that you had to run them again 3/4 of the time. Your new probability is 9%, only about 1 in 11. Since that's 3/4 versus 1/11 of all test runs, including the ones you did as repeats, that means a drastically shortened chain of tests and retests, with shorter links and vastly fewer of them.
Now let's finally pile on all the other benefits. Of course better separation of concerns would lead to cleaner design, for easier extension, maintenance, and repair. But it would also ease some of the other pains we were having. If people were devoted to just some particular portal, and maybe permitted to mess with the share code, they could separate the backlogs, and the source repositories, and so on. That means only 20-40% as much email flooding their inboxes about the latest feature additions, bug reports, pull requests, and so on. They might even turn their email notifiers back on, so they can find out about real emergencies in a reasonable amount of time. They might be less tempted to skip testing. They might be able to develop deep expertise on a piece of the project, instead of knowing just enough about all of it to be dangerous. They might not abandon the project (or the entire company) in frustration. And on and on.
So now it's your turn, dear reader. Have you worked on a megalithic app, with obvious seams to break it apart at? Did you do it? If so, what benefits did it bring -- what pains did it ease, what pleasures did it bring? If not, why not? Either way, what eventually happened?
An app composed of four clearly defined parts could be written as four, or at least three, separate Rails engines mounted on one app. Or possibly as four separate apps with a library of shared code, all accessing the same database. Or if the four parts are relatively small, one application that includes them all. In this case... it was monolithic, but not of small parts at all. We repeatedly recommended breaking it up, but the client just didn't want to. Funny thing about consulting is, the client is always right -- not just even when he's wrong, but especially when he's clearly, provably, absolutely flat-out dead wrong.
"So what?", you might be wondering. "What good would breaking it up do, aside from pleasing the ivory-tower purists?"
The problem is, this has quite an effect on the rate of progress. Even if the engineering of the code itself is very clean, so you don't have god-objects causing churn and interference in the actual code (as the client did), and you have excellent developers making rapid progress in developing features (as, thankfully, both the client and we did, he said ever so humbly)... look at what happens when someone's just trying to commit a feature. The skill level of the people involved doesn't matter at all at this stage, it's just pure statistics.
Suppose you are on a team of about twenty developers, making a web app to run a school, with different portals for teachers, administrators, students, and parents. You and a colleague pair program on the next highest-priority ticket from the backlog in the issue tracker. It turns out to be in the parent-portal. You create a feature-branch, off the master branch. You get the feature passing the automated acceptance tests given in the user story, pull the latest master, merge it into your feature-branch (resolving any conflicts), and run the whole test suite. It passes, so you didn't break anything.
Now what? Merge into master? You try... and get rejected by the revision control system because someone committed changes to master while you were testing. Not surprising, as there are twenty of you, all hard at work, and maybe some of them aren't pairing so there are even more than ten features under active development... and the test suite takes an hour to run. The changes were in a different portal, so unlikely to interfere... but better safe than sorry. Nobody wants to be the jerk who broke the build, or even the Continuous Integration server. So you pull the latest master, merge it into your branch, and run the test suite again. Still green, so your changes don't break anything that was merged into master during your previous test run. Nice to know, but it means that one of the test runs was a total waste of time. Maybe you used it for something else productive... or not. Either way, it's nearly certain that your mental house of cards concerning that feature has utterly collapsed, because you've been thinking about something else.
So now, insert the above paragraph, again. Maybe you pull some fancy tricks like parallelization and extensive mocking and stubbing of slow services and expensive object creation and so on, and cut the time down to fifteen minutes. That's still plenty enough time for it to happen again. And again. Lather, rinse, repeat, ad nauseam, which gets to feeling like ad infinitum... especially to your client, who is breathing down your neck, waiting for this feature.
Nobody's happy, not him, not your boss, and certainly not you. You're a good developer, you want to be productive, you believe in testing... but you hate the inane futility of having to do it over and over in vain.
Now let's consider what it might be like if the app had been broken up into separate engines or apps. Even assuming a fifth item (an app to mount engines on, or a library of code shared among apps), the size of the codebase you need to deal with at one time is still cut down to 40% (assuming all pieces are of equal size). You could possibly make it 20%, but let's even assume the worst (within the assumptions already made). How would that help with this problem?
First, only 20 to 40% of the changes being worked on (assuming equal distribution of current work) are likely to block your code from being accepted by version control. (You're probably only working on one of the apps/engines, and maybe the library or main app but usually not.) Call it 30% to keep it simple. This means only 30% of the probability that any given change, to the overall system, will make you run the test suite again.
But wait! There's more! (Or rather, there's less!) The time it takes to run the test suite, should also get cut to about 40%. (Less when you don't change anything the shared code depends on (so you only have to run one portal/app's tests), same when you do (that plus the shared code's tests), more when you change the shared code (everything). Call it a wash to make the math easy. Still certainly a smallish fraction.)
Not only is that an instant time-savings right there, and not only does it make it easier for you to stay mentally on-task in case some more changes are needed, but:
Combine these two factors, and they mean you have only 12% of the original probability of an interfering change happening during your tests, making you run them again. Suppose the original probability was 75%, so that you had to run them again 3/4 of the time. Your new probability is 9%, only about 1 in 11. Since that's 3/4 versus 1/11 of all test runs, including the ones you did as repeats, that means a drastically shortened chain of tests and retests, with shorter links and vastly fewer of them.
Now let's finally pile on all the other benefits. Of course better separation of concerns would lead to cleaner design, for easier extension, maintenance, and repair. But it would also ease some of the other pains we were having. If people were devoted to just some particular portal, and maybe permitted to mess with the share code, they could separate the backlogs, and the source repositories, and so on. That means only 20-40% as much email flooding their inboxes about the latest feature additions, bug reports, pull requests, and so on. They might even turn their email notifiers back on, so they can find out about real emergencies in a reasonable amount of time. They might be less tempted to skip testing. They might be able to develop deep expertise on a piece of the project, instead of knowing just enough about all of it to be dangerous. They might not abandon the project (or the entire company) in frustration. And on and on.
So now it's your turn, dear reader. Have you worked on a megalithic app, with obvious seams to break it apart at? Did you do it? If so, what benefits did it bring -- what pains did it ease, what pleasures did it bring? If not, why not? Either way, what eventually happened?
Tuesday, October 2, 2012
Heroku for Staging and Production, Without Confusion
When you want to put up a Ruby on Rails app on Heroku, chances are you configure it so that when you deploy, you use the classic
When might you not want to do this? Mainly when you don't want to bother having some other URL for the production site. I've still got a few little apps that are addressed by their herokuapp.com URLs, like rebalancR (http://rebalancR.herokuapp.com) and The Decider (http://TheDecider.herokuapp.com). Using those URLs just looks ugly. In these cases I didn't really care. The first was a prototype for a client. The second was basically an early proof that yes I could do Rails even though I didn't yet have any public RoR projects I could show.
However, the one with the above git config file, is also available as http://TheJobsmith.com. In fact, my main web site (http://www.codosaur.us) is technically a Rails app -- and also available as http://codosaurus-production.herokuapp.com, with a staging site at http://codosaurus-staging.herokuapp.com (and a git config file very similar to the above).
git push heroku masterRight? But... with a little change, you can make it easy to deploy to both staging and production versions, both on Heroku, with little to no confusion, as can happen if you still refer to one or the other as heroku. Here's the .git/config of one of my projects, with the important bits in bold:
(Of course, this is after specifying or changing the names, to be $APP-staging and $APP-production.) Note the [remote "staging"] and [remote "production"] parts. When I want to push to staging, it's:[core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true ignorecase = true [remote "staging"] url = git@heroku.com:jobsmith-staging.git fetch = +refs/heads/*:refs/remotes/heroku/* [remote "production"] url = git@heroku.com:jobsmith-production.git fetch = +refs/heads/*:refs/remotes/heroku/*
git push staging masterand a push to to production is:
git push production masterNo need to worry about whether just plain heroku is staging or production.
When might you not want to do this? Mainly when you don't want to bother having some other URL for the production site. I've still got a few little apps that are addressed by their herokuapp.com URLs, like rebalancR (http://rebalancR.herokuapp.com) and The Decider (http://TheDecider.herokuapp.com). Using those URLs just looks ugly. In these cases I didn't really care. The first was a prototype for a client. The second was basically an early proof that yes I could do Rails even though I didn't yet have any public RoR projects I could show.
However, the one with the above git config file, is also available as http://TheJobsmith.com. In fact, my main web site (http://www.codosaur.us) is technically a Rails app -- and also available as http://codosaurus-production.herokuapp.com, with a staging site at http://codosaurus-staging.herokuapp.com (and a git config file very similar to the above).
Tuesday, September 25, 2012
Free Heroku Text Search Update
I volunteered to give a talk on "Full Text Search on Heroku for FREE", subject of two prior blog posts (here and here), at Arlington Ruby Meetup. So, I've prepared slides, which are available at https://docs.google.com/presentation/d/1NbN0kJMJsSQW2N7ItNMB6VuM_lJok-Xb0epk0anbRIo/edit . The slides contain code samples (as text, not screenshot graphics) that you can copy.
UPDATE: I gave the talk at RubyDCamp, and it was well received.
UPDATE: I gave the talk at RubyDCamp, and it was well received.
Saturday, September 22, 2012
Domains Shuffled!
The domains seem to be all in order. I don't know if this will automagically get to all the subscribers... this post will also be a test. :-)
Do the Domain Shuffle!
Heads up! I've finally pulled the proverbial trigger on setting up my LLC. Henceforth, Dave Aronson Software Engineering & Training will be Codosaurus, LCC, and will take over this blog's domain, http://www.codosaur.us. This blog is about to move to http://blog.codosaur.us. http://www.davearonson.com will eventually be replaced with stuff more about me personally.
Subscribe to:
Posts (Atom)