Brian Ghidinelli interview on Cars Yeah

It was a lot of fun! Mark has interviewed hundreds of interesting business and motorsport personalities and to be included on the list is an honor. Be careful – you might spend all day listening!

This year I’ve been working to do more A/B testing. A/B tests are mathematically-backed competitions between two or more options which are scored by the actions of the users. They may also be the single highest ROI tactic for software companies. These tests could be anything from a simple color change of a button (shocking, to what extent that can matter) all the way to a completely different web page. I was first exposed to the technique back in 2001 at Yahoo! when I was helping redesign their search results but it wasn’t until the last 18 months I ran the first A/B test on MotorsportReg.com.

Which brings me to how I grew one of my mailing lists by 53% in 45 days. In the history of this 9-year old list, the last two months would make a hockey stick look like a rolling foothill. This is like the face of a glacier where it meets into the ocean: a vertical line that reads like an error.

The technique is perhaps too simple to be valuable by itself: I put the signup form where many more people would see it in their routine use of our platform. Previously, we had a few CTAs in our transactional emails and on our event calendar but the actual signup took place on a mailing lists screen under “My Account”. Few people went there looking to add more email to their inbox so list growth was steady but slow. Now, when users create a new account or reconfirm their details, they see checkboxes at the bottom of their personal information letting them opt-in to the lists. Thousands of people see these screens every month and because the lists are valuable, thousands of them are signing on.

“It was just a test”

The obvious question is why didn’t we do this sooner? The answer is because I wasn’t sure if these transactional flows were appropriate places to insert a list signup. I had concerns that it might negatively impact registration conversion. So I hemmed. And I hawed. And I periodically looked at the issue in our bug tracker but never quite got over the “ewwwww” feeling to put it in place.

That’s the beauty of A/B testing. A feature request in our bug tracker is a stone being added to a wall with mortar: it becomes a seemingly permanent piece of your architecture, of your user interface and of your responsibility. But an A/B test, why, that’s just a test! We have made no commitments to keeping it around. We’re not even sure we like it! Implementation only took a few hours so we could rip it out at any time for any reason and respond, “It was just a test.” Simply doing it, rather than talking about it, is rule #2 of A/B testing fight club.

Since it was just a test, we hoped for a nice bump to justify keeping it around. We were blown away. At the current rate, we will reach 25,000 subscribers (333% growth) by the end of 2015. And because we have a large percentage of first-time participants come through our service, we should see a permanently scaled growth rate. As an additional bonus, this list drives participation for our events so we’re creating a positive feedback loop for our event organizers and our bottom line too.

All because of a feature I was skeptical of “implementing” but happy to “test”.

Remember: making the call is making progress. Doing is better than planning. Execution is more valuable than ideas. Look at your to-do list and find one or two things you’ve been putting off. What can you do to “test” it (whatever that might mean in your case) to move it ahead and gain confidence in your choice?

Pic linked from wren.

My favorite question was “What are you most excited about, auto racing, travel or startups?” See what I said.

CFPAYMENT makes payment processing easy. It’s a ColdFusion library modeled after Ruby’s ActiveMerchant that normalizes various payment gateways to give developers a single interface to process credit card and ACH transactions. After years of production use and tens of millions of dollars processed, I’m proud to announce a 1.0 release is now available at cfpayment.riaforge.org.

There’s a lot of hard work from many different folks that have made this a success. If you’re interested in learning more, download the file and check out docs/cfpayment.pdf or hit up the Google group at http://groups.google.com/group/cfpayment.

While Batchbook does have the pretty cool web forms which can capture contact data from any ole web form, it doesn’t give you total flexibility with filling in customized data fields. In our case, we wanted to create companies rather than individuals as part of a sales pipeline so we needed to have more control than the web forms currently allow.

Start with a contact form

Here’s the HTML form that we’re using – it still uses the original web form field names so all I’ve done here is changed the form ACTION to point at my CFM instead of Batchbook:

<form method="post" action="https://ourserver.com/form.cfm">
<input type="hidden" name="location[address][country]" value="US" />
<h2>Sign-up Now!</h2>
<table class="form">
<tr><td><label>Company *</label></td></tr>
<tr><td><input type="text" name="company[name]" size="30" /></td></tr>
<tr><td><label>Account Type *</label></td></tr>
<tr><td><select name="supertags[sales first contact][plan]"><option value="">Select a plan - you can change later</option><option value="Plan 1">Plan Uno</option><option value="Plan 2">Plan Dos</option><option value="Plan 3">Plan Tres</option></select></td></tr>
<tr><td><label>First Name *</label></td></tr>
<tr><td><input type="text" name="contact_details[first_name]" size="30" /></td></tr>
<tr><td><label>Last Name *</label></td></tr>
<tr><td><input type="text" name="contact_details[last_name]" size="30" /></td></tr>
<tr><td><label>Email *</label></td></tr>
<tr><td><input type="text" name="location[email]" size="30" /></td></tr>
<tr><td><label>Phone *</label></td></tr>
<tr><td><input type="text" name="location[phone]" size="30" /></td></tr>
<tr><td><label>Organization Address</label></td></tr>
<tr><td><input type="text" name="location[address][address_1]" size="30" /></td></tr>
<tr><td><label>Address 2</label></td></tr>
<tr><td><input type="text" name="location[address][address_2]" size="30" /></td></tr>
<tr><td><label>City, State Zip</label></td></tr>
<tr><td><input type="text" name="location[address][city]" size="15" />, <input type="text" name="location[address][state]" size="4" /> <input type="text" name="location[address][zip_code]" size="10" /></td></tr>
<tr><td><label>Company URL</label></td></tr>
<tr><td><input type="text" name="location[website]" size="40" /></td></tr>
<tr><td><label>Do you have an existing service?  If so, which:</label></td></tr>
<tr><td><input type="text" name="supertags[sales first contact][existing_service]" size="40" /></td></tr>
<tr><td><label>Date of next event:</label></td></tr>
<tr><td><input type="text" name="supertags[sales first contact][date_of_first_event]" size="15" /><br /><small>Format date like mm/dd/yyyy</small></td></tr>
<tr><td><label>Questions/Comments</label></td></tr>
<tr><td><textarea name="supertags[sales first contact][customer_comments]" rows="10" cols="50"></textarea></td></tr>
<tr class="submit"><td><input class="button" type="submit" value="Request Account" /></td></tr>
</table>
</form>

It’s the same contact form you’ve whipped up 100 times before. If you’re using the built in web forms, the field names must match what is above. If I was writing my form handler from scratch I would have selected more normalized names.

Note that we a hidden field at the beginning – not all of the data must be user editable. Of course, I could also just set it in my form handler. The country value is a holdover from the original web form.

Process those fields

Our next step is to process that form submission with ColdFusion and use the Batchbook API to create my contacts and populate my custom data fields:

< !--- credentials --->
<cfset variables.api_key = 'YOUR-SECURITY-KEY' />< !--- get this from "Your Account", right column --->
<cfset variables.root_uri = 'https://[YOUR HOST].batchbook.com/service' />

<cfhttp url="#variables.root_uri#/companies.xml" method="post" username="#variables.api_key#" password="x" charset="UTF-8" timeout="30" throwonerror="no">
	<cfhttpparam name="company[name]" value="#form['company[name]']#" type="formfield" />
	<cfhttpparam name="company[notes]" value="" type="formfield" />
</cfhttp>
< !--- cfdump var="#cfhttp#" --->

<cfif structKeyExists(cfhttp, "responseheader") AND isStruct(cfhttp.responseheader) AND structKeyExists(cfhttp.responseheader, "location")>

	<cfhttp url="#cfhttp.responseheader.location#" method="get" username="#variables.api_key#" password="x" charset="UTF-8" timeout="30" throwonerror="no">
	</cfhttp>
	< !--- cfdump var="#cfhttp#" --->

	<cfset xmlCompany = xmlParse(cfhttp.fileContent) />
	<cfset id = xmlCompany.company.id.xmlText />

	< !--- now add location to company --->
	<cfhttp url="#variables.root_uri#/companies/#id#/locations.xml" method="post" username="#variables.api_key#" password="x" charset="UTF-8" timeout="30" throwonerror="no">
		<cfhttpparam name="location[label]" value="work" type="formfield" />
		<cfhttpparam name="location[email]" value="#form['location[email]']#" type="formfield" />
		<cfhttpparam name="location[website]" value="#form['location[website]']#" type="formfield" />
		<cfhttpparam name="location[phone]" value="#form['location[phone]']#" type="formfield" />
		<cfhttpparam name="location[street_1]" value="#form['location[address][address_1]']#" type="formfield" />
		<cfhttpparam name="location[street_2]" value="#form['location[address][address_2]']#" type="formfield" />
		<cfhttpparam name="location[city]" value="#form['location[address][city]']#" type="formfield" />
		<cfhttpparam name="location[state]" value="#form['location[address][state]']#" type="formfield" />
		<cfhttpparam name="location[postal_code]" value="#form['location[address][zip_code]']#" type="formfield" />
		<cfhttpparam name="location[country]" value="#form['location[address][country]']#" type="formfield" />
	</cfhttp>
	< !--- cfdump var="#cfhttp#" --->

	< !--- set default values for sales super tag and capture user-provided information --->
	<cfset arrSuperTag = arrayNew(1) />
	<cfset arrayAppend(arrSuperTag, "super_tag[existing_service]=#URLEncodedFormat(form['supertags[sales first contact][existing_service]'])#") />
	<cfset arrayAppend(arrSuperTag, "super_tag[date_of_first_event]=#URLEncodedFormat(form['supertags[sales first contact][date_of_first_event]'])#") />
	<cfset arrayAppend(arrSuperTag, "super_tag[customer_comments]=#URLEncodedFormat(form['supertags[sales first contact][customer_comments]'])#") />
	<cfset arrayAppend(arrSuperTag, "super_tag[plan]=#URLEncodedFormat(form['supertags[sales first contact][plan]'])#") />
	<cfset arrayAppend(arrSuperTag, "super_tag[requested_demo]=#URLEncodedFormat(form['supertags[sales first contact][requested_demo]'])#") />
	<cfset arrayAppend(arrSuperTag, "super_tag[active]=true") />
	<cfset arrayAppend(arrSuperTag, "super_tag[paperwork_sent]=false") />
	<cfset arrayAppend(arrSuperTag, "super_tag[agreement_back]=false") />
	<cfset arrayAppend(arrSuperTag, "super_tag[announced_on_facebook]=false") />
	<cfset arrayAppend(arrSuperTag, "super_tag[buddy_check_complete]=false") />	

	<cfhttp url="#variables.root_uri#/companies/#id#/super_tags/sales.xml" method="put" username="#variables.api_key#" password="x" charset="UTF-8" timeout="30" throwonerror="no">
		<cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded" />
		<cfhttpparam type="body" value="#arrayToList(arrSuperTag, "&")#" />
	</cfhttp>
	< !--- cfdump var="#cfhttp#" --->

	< !--- create person--->
	<cfhttp url="#variables.root_uri#/people.xml" method="post" username="#variables.api_key#" password="x" charset="UTF-8" timeout="30" throwonerror="no">
		<cfhttpparam name="person[first_name]" value="#form['contact_details[first_name]']#" type="formfield" />
		<cfhttpparam name="person[last_name]" value="#form['contact_details[last_name]']#" type="formfield" />
		<cfhttpparam name="person[company]" value="#form['company[name]']#" type="formfield" />
	</cfhttp>
	< !--- cfdump var="#cfhttp#" --->
</cfif>

Some notes about the above:

• What’s my password? Your password is X. Or Y. Or Z. It doesn’t matter – Batchbook only authenticates you on your unique key which is sent as the username. The password can be anything.
• If you’re new to REST APIs, I suggest enabling the CFDUMPs as they will show you how data comes and goes. Plus, you’ll see what the response headers look like (hint, they aren’t all 200 OKs).
• Associating people with companies – there is no official way to do this. Just make sure the “company” value for the person is a string match for the company record and Batchbook will make the magic happen on their end.
• There is basically no error checking or exception handling here – I’ve got my code wrapped up in some try/catch and I fall back to sending an email to us if all else fails. Plan for failure. At some point the API will be down or the Internet will break and you need to have a contingency plan when dealing with remote third parties.

The above code took me a day or two of toying around to get working properly. While the docs are pretty good, my experience is that API implementations never quite match their documentation. Getting super tags to work (key to our sales process) took a lot of fooling around plus some assistance from the very helpful Eric Krause at Batchbook.

Next post will be how we’re integrating the results of this sales pipeline with Email Center Pro to dynamically generate and send out contracts in preparation for our sales cycle this winter.

Don’t let a little bit of hate make you second guess yourself. Success usually comes at the end of a very long road of perseverance.

That’s why I was surprised to get an email this morning from my buddy Trevor Peace who found our project for the 2005 DARPA Grand Challenge not just online but actually still functioning! Fun to reminisce!

We built a real-time Flash leaderboard that showed where the automated vehicles were on course from a GPS location feed. This was one of the few projects that I worked on that was an “event”. E.g., it wasn’t launched and then took on a life of its own. Instead, the shelf life for this was just a few weeks with a particularly important 36 hours during the race through the desert near Primm, Nevada. That kind of pressure brings new emphasis on testing. Especially when your customer is the Department of Defense!

The lack of permanence on the web makes me envious of people who work with their hands for a living. I’m sure this is a “grass is greener” feeling but the kind of work where you physically create or sell something has a lot of appeal to me at this stage of my life. It’s also a lot easier to explain to your grandparents.

Why would you expect someone to provide a higher level of availability and more features at a cost lower than you can do it yourself?

You can argue about economies of scale but fundamentally it’s on the mark. Amazon (and the other grid/cloud providers) have to pay for the same hardware and infrastructure that we do except they have even more strenuous requirements. And since they’re not a charity, there has to be some profit in there, too.

But many of the reasons to wait on cloud computing, at least the Amazon variety, changed today with the announcement of three new services:

• CloudWatch – Web services-based resource and performance monitoring of EC2 services at $0.015/hr or $10.80/month.
• Auto Scaling – A key feature of add-on services like RightScale, Auto scaling will add or remove server instances on the fly. Most web sites experience some amount of seasonality whether it be time of day, day of week or month of year and this will reduce costs by scaling up only as needed. It’s included as part of CloudWatch.
• Elastic Load Balancing – Yes! Instead of setting up an instance and running keepalived or haproxy to route traffic between multiple servers, Amazon now has load balancing at $0.025/hr or $18/month and it can span multiple availability zones (~physical data centers). You also pay $0.008/GB of data transfer handled by the load balancer. 100GB/month would cost you $0.80.

Why is this news?

It means EC2 just got cheaper as you scale up.

For my company, we run a two-node cluster primarily for high-availability. To actually keep this highly available, we also have to run a pair of load balancers managed by keepalived. This works quite well but it’s 4 separate boxes with power supplies and disks consuming power and taking up rack space. Those are all the downsides to having your own rack space. At least in California, power is the most expensive part of physical hosting and also the biggest restriction. It took a fairly exhaustive search to find a datacenter where we could get 30A into a single cabinet.

If we had our current architecture in EC2 today, my interpretation is we could eliminate our two load balancers to save $144/month (small instances are ~$72/month) and replace them with $11 worth of Elastic Load Balancing. If we added monitoring, we should be able to add web servers as demand dictates for additional savings by not having a second web server running 24/7.

Most smaller web companies want five nines but can’t afford it nor manage it. Historically we run our service just short of 99.99% uptime which is fairly expensive and difficult to achieve but provides a scant 3.5 hours of downtime per year. Most companies can tolerate this. These new features of EC2 are making it inexpensive to take advantage of EC2s promise: on-demand scalability with high-availability.

How to save money with EC2

When I agreed it was insane to expect someone else to do something for you for less money, it was if you compare apples to apples. Everyone has a base architecture that has something like a database server, web server and mail server. You simply cannot expect to save much, if any, money in your initial configuration. The only meaningful way to save money with cloud computing is to reduce the incremental step in costs as you add capacity. If a colocated solution offers you additional 10 units of web server performance for $1000/month and you can buy the same performance in 1-unit increments for $100/month, then you save money anywhere between 10 and 20 units. Virtualization and auto-scaling make it possible to reduce your investment to the minimum required capacity on a minute-by-minute basis to drive the most value from cloud hosting.

I think it’s time to start running some tests…

• The erroneously named Web application autopsy is actually a recap of a SXSW panel looking at four different companies at four different stages. The interesting bit here is each company, Wufoo, Blinksale, Feedburner and RegOnline each shared some hard numbers from their operations.
• From there I read through to Underbelly of a web app. We have a lot of this stuff already taken care of – the billing, customer service and other automation pieces that make the actual operations of a company manageable.
• The link about stat tracking caught my eye and pointed to Joe Kraus’ Confessions of a startup addict. This is not new by any means but I realized that we weren’t doing a very good job of tracking revenue by customer so I dropped into Eclipse and threw together a Statistics Service for my app that will generate this data for me broken out by plan and date. Having metrics to gauge progress (and more importantly, motivate) is key to success but so easy to ignore.
• Someone in one of these pages referenced StartupSchool.org, which is supposed to happen again in 2009. I found a link to videos of their talks which I’m going to watch this weekend for some inspiration.

Got any good business / startup / entrepreneurial readings? I also came across Bill Flagg’s blog when I went looking for more info about RegOnline and the guy who participated in the panel. It’s no surprise given my company is MotorsportReg.com that I would be interested in the success of RegOnline.com. I found Bill’s blog to be frighteningly similar to my own thoughts about the web, applications, customer service and so forth. We seem to read a lot of the same books as well but what I found valuable about his posts were the amount of actual data and insight he shared from RegOnline’s operations.

P.S., when are you no longer a “startup”? Time? Revenue? Ever?