A Testable PayPal IPN with Rails 2.3

I’ve been building out an IPN (PayPal’s Instant Payment Notification) responder using Rails 2.3.5 recently and managed to get it to work with Mongo, Cucumber, and Pickle without mocking and by testing pure form posts to the IPN in an automated way using RSPec’s post method. That’s a mouthful – but here’s what I have.


Quick Background

For those who have never heard of the IPN – the basics are that you can have someone pay for something with PayPal and PayPal will “ping” your site with a form request that details the transaction. You write some code that handles the form post and validates it with PayPal, and off you go.

I know there are a number of people who really don’t like working with PayPal – and also paying with PayPal. For some it’s extremely helpful to have a payment system in place in a matter of hours versus days, and to have a fee structure that allows you to pay per transaction. This post isn’t about PayPal – it’s about how to use the IPN if you find yourself in the position of having to support it.

As you can imagine, testing this kind of interaction can be a bit cumbersome given the amount of interaction required with PayPal. Typically people have just written the code, created a form on their site and submitted arbitrary data to see the result, or have used 3rd party tools. You can also expose your IP and port 80 so PayPal can ping your IP address and you can view a log of the result.

All of this is not terribly optimal, and is also ridiculously slow. The good news is that Ruby and Rails make this almost painless if you use Cucumber and RSpec.

Getting Started

I know that ActiveMerchant will do much of this for you – the problem is that it didn’t do all of what I needed at the time (and I also needed the gem for only one thing) so I followed their lead (and borrowed some of the code) and made my own.

I should also mention that if you want the code, I put it up on Github.

The first thing to do is to create your site and setup Cucumber. I’m using Rails 2.3.5 at this time, and you’ll need to have the cucumber, cucumber-rails, rspec, and rspec-rails gems installed:

rails paypal_demoscript/generate cucumberscript/generate rscpec

Now that the test structure is in place, crack open config/environment.rb and set some constants (again, borrowing these settins from ActiveMerchant). This isn’t so much for testing as it is for when you run your site locally and want to use the Sandbox to triple check everything…envsettings.png

The PayPalNotifcation Class

There are a number of ways to handle this process – you could, if you wanted, just leave everything in a PayPalController and have it read in the post values coming from PayPal via the IPN. You *could* do this, but it can get more than a bit ugly, and real quick.

Before we dive into the code, it’s important to understand what’s happening:

  1. PayPal pings our site and says "someone bought something"
  2. The ping comes in as a POST, with a ton of form values for us to read and consider
  3. Given that this is the web, people will try to spoof this and expose your system, so PayPal asks that you ping them back to verify that what you’ve been handed is indeed valid.
  4. The return ping to PayPal is, once again, a POST – but this time over SSL.
  5. The response from PayPal will be a single value – "VERIFIED" or "INVALID". All we really care about is "VERIFIED" – that means we have $$.

As you can see there’s a lot going on here – so it’s probably better just to give this over to it’s own model. In this case I’ve called it PayPalNotification. It takes in a hash (form posts in Rails are passed around as a hash) which represents the form post, and returns various values that you might need (including an array for the item number and item names)
PayPalNotification.png

(The rest of the code is up at Github in a Gist)

The Controller

As I mention before, I had thought about implementing a PayPalController, but it didn’t feel right. They can and do have logic in them, but they’re supposed to control the flow of the app, pushing/pulling data from the view. Instead I opted for a regular PaymentsController that exposed a method just for the IPN. In this way I could re-use the Controller if they client ever needed to use a different payment process (which they will, many people won’t use PayPal).
PaymentsController.png

In this code we create a new instance of the PayPalNotification and pass off the form params. The most important line in there is "notify.acknowledge" – which fires this code in the PayPalNotification class (not seen above) – once again this is almost verbatum from ActiveMerchant:
notify_ack.png

The part that I added was the conditional for RAILS_ENV – if it’s a test I don’t want it to call back to PayPal. Instead I just want it acknowledged. I had thought about an override for when it’s not acknowledged (INVALID) – but if that’s the case then nothing happens at all, so I’m OK leaving it like this.

The Test

I’m using Cucumber to test this – but you can use RSpec or Shoulda just as easily. The first thing I need to do is to construct the specifications. These are simplistic, but I need them to make sense in the context of this post so please don’t look too deeply at whether the naming is valid :) . I have a lot more specs, but here are a few to get you started:

paypal_specs.png

These should probably be done up as Unit Tests or more concise RSPec specifications, but I really like the way Cucumber makes my brain work – for some reason testing and being able to write scenarios in my native tongue takes my mind out of the code and into the business needs.

To make these specs pass, I need to construct an IPN based on the passed-in information. I’ll quickly go on a tangent here for people who aren’t familiar with Cucumber – it uses a DSL called "Gherkin" (which is what you see above) and is able to parse things from it – like strings and numbers. It then tries to match each of the "Given/When/Then/And/But" statements (called "steps") to Ruby methods that you’ve written. You can then work with these parsed sentences and have them execute. It’s a very handy system.

To support the first step "Given an incoming IPN for 4 items" I created a single step, with a supporting method:
step1.png

This "step" is defined in a separate file, and is pure Ruby code. The above is a code block, that evaluates a regular expression and opens up a block to work with that expression and the matches that come out of it – which in this case is the number of items I want to see in the IPN.

I then set a class-level variable (so all my tests can see it) by calling a rather long method, that simply creates a hash for me. I can use that hash as my form post later on…step_1b.png

<snip>
step_1c.png

At the very end is where I merge in the form values for inspection later. For more information on what these values are and represent- have a look at the PayPal IPN variables page.

Now that I have my form hash, I’m ready to roll with the rest of the steps:
rest_of_steps.png

If you’re working in TextMate and you’ve pointed it to your shell, you can hit Command-R and run the specs right from TextMate. Running these now, I see this:
passing.png

This is all fine and dandy, but it doesn’t really capture the behavior of the application. What I’d really like to do is ping my own site, sending in the IPN values for the Controller to process.

This is why I really dig Cucumber/RSpec/Webrat.

Testing The Controller

How you integrate the IPN can vary based on what your site does. A typical site might save an order to the database on checkout as "Unpaid", then wait for a payment notice to come in which will tick the status of the Order and also record a Transaction record. There are variations, of course, and every site does what the business requires. But I’ll use this as a scenario right now. Here’s a basic specification:
controller_spec1.png

Fans of the Pickle framework will recoginize the way some of these steps are written – Pickle contains hooks into pre-written steps that will pull or push data to your database if you write your steps a certain way. In the first step there, I’m specifying my model "order" and by saying it "exists" Pickle will push it into the test database for me. If I continue on and add "with status: ‘Unpaid’" Pickle will be sure to set that on the model item for me. Likewise, a few steps below I ask for the model back with "should exist" but this time Pickle uses the "with" as criteria for pulling the model back out. Good stuff – keeps me from writing a lot of steps.

To get this work work, I need to write a step for "When an IPN comes in for that order". I can use the same hash code as before, pushing in the data that I need to from the order that’s in my test database (I can ask for order.first if I like – there’s only one in there as the database gets cleaned out ever run).

Order set, hash created, all I have to do is add one line:
controller_step1.png

[UPDATE] – I originally wrote that an HTTP post was fired here – turns out I was wrong. It’s fundamentally the same thing, but the operation happens inside of Rails – a request is not made to your Rails app.

The last line is "post" – which fires off an simulated post to a given URL – loading your controller and executing an action as if it was a Rails invocation. I could use Rails named routes here, but it won’t work unless I go through Cucumber’s routing features – which I don’t want to do. I’m OK keeping it hard-coded, which I think could get frowned upon – truth is I don’t really know a cleaner way to do it :) . If you do, please let me know.

This will post my hash to the ipn method on my controller, which contains the logic that will, hopefully, update the order and set its status properly. It’s worth noting here that Rails tests gladly and willingly hit a special test database (not development or live production) so it becomes a pretty simple matter to check the test system to see things got updated as you expect.

In this case I’m just using Pickle, so I don’t need to fire up additional steps to verify the order has been updated.