Kill Your Users Table
Thursday, July 03, 2008 -
It's time to rethink how we store user information. You don't need as much as you think you do.
Consider...
Reddit's database was stolen a little over a year ago (yes, I know, old news. There's a point here). This, from the team blog (emphasis mine).
...a backup of a portion of the reddit database was stolen recently. Although the media did not contain any personally identifiable information about our users ... we wanted to alert you to the possibility that your username, password, and in some cases e-mail address may have been compromised. If you use the user name and/or password for other purposes, we suggest that you change them in those other uses as soon as possible - just in case.
This is a response from spez, one of the developers.
[Password encryption] is [easy to implement], and I'll go ahead and do it now that everyone has decided to weigh in.
Personally, I prefer the convenience of being having my passwords emailed to me when I forget, which happens from time to time since I use difference passwords everywhere.
Not hashing was a design decision we made in the beginning, and it didn't stem from irresponsibility-- it stemmed from a decision to provide functionality that I liked.
It bit us in the ass this time, and we are truly sorry for it. The irresponsibility (and there is some) was allowing our data to get nabbed.
We can beat on the Reddit guys for this all we want, however (in lighter form), applications still retain a lot more user information than they used to and perhaps it's time to change that.
Think Like a Hacker
Mr. Savvy Hacker knows that it's not too hard to imagine a way to get at some good information with the data from Reddit:
//loop the millions of users we just stole //and see if we can login to their email foreach(User u in RedditHackedDatabase){ string email=u.Email; string password=u.Password; //setup login spoof for web-based emails, like //Google, LiveMail, Yahoo, etc //.... }
If we assume that Reddit has a million users, odds are that some percentage of those users would have the same password as their email address. It happens -it's people mechanics and people are too trusting.
Let's say the net of this harvest is small - 1000 emails compromised. It took maybe 4-6 hours to run this looping routine, and right now Reddit doesn't even know their data is gone. Once they find out, they'll probably run damage control which will take another 6-8 hours, and then they'll let their users know. The Savvy Hacker has a day or more to go nuts with these email accounts.
Things like looking for forgotten password emails. From your bank.
Rethink It
Today I was on a call with Scott Hanselman who is leading a charge to re-imagine and redo the Northwind Database. This was a Skypecast and we talked about all kinds of things, and he asked me about my experiences with Membership and the MVC Storefront.
I suggested we kill the Users table.
This devolved into some back and forth where my thoughts were challenged greatly (by many) and I can sum the challenges thus:
- The application needs to at least know the first/last of the user so I can recognize them, at the very least. All sites do this.
- What about [featureX]! There's no way I can do this without knowing who the user is
- How's my application going to send information to the user
These are very good points, and I think that there is room here to rethink "how it's been done" and come up with a way that protects our users better. Perhaps we can start by understanding this information is not something we need in our system, in just about every case.
User Identification As A Service
Enter services like Open ID and Passport/LiveID. These services will store information about the User for you. You don't need it. Really.
There are some things where you might want to know a user's first/last name - but ask yourself why. Does the application really need it?
- Billing and Shipping Addresses: Billing records are kept at the payment processor, and shipping is likely kept in a waybill somewhere - both accessible by your application when needed. You don't need this in your database.
- Names of people who review/comment in my application: You can get this using OpenID and LiveID, or you can ask for a reference when they add the review/comment. Scott's blog is a great example of this.
There are always exceptions to this (social applications - like forums); even then there are ways to minimize what you know about your users. The point about the application contacting the user is a good one, but I might suggest using an "Opt-in" - where you ask the user if they want to be contacted. I never do - but I always have to turn that feature off.
Information storage is a trend, however, and it's often easier to "just ask" so we have it when we need it; in other words we opt for convenience. This design philosophy echoes what spez (Reddit Developer) said above:
Not hashing was a design decision we made in the beginning, and it didn't stem from irresponsibility-- it stemmed from a decision to provide functionality that I liked.
This design decision wasn't thought out all the way. The ramifications clearly not fully understood. It's time to change this way of thinking.
Ultimately people will think that the responsibility sits with the user to provide a unique password and "take care of themselves". This is a comment on the Reddit site, after spez's above:
Access credentials are a mutual decision (I pick my username/password, you store them), so it is up to both of us to decide how important it is that someone can't pose as me or access my data on this particular website. If I'm particularly worried about these things, I'll pick a password that is hard to break, and ask you about how you store passwords. If I'm not particularly worried, I'll just use a different password than the sites I am worried about people breaking into.
This is a reasonable point, although I might argue that it's usually pretty darn important that users don't pose as other users - else what's the point? Perhaps notifying the user is something we could do:
When you login to our site, please be aware that we don't take strong measures to protect your data and some or all of your information may be compromised at some point in the future if our data is stolen. As such, we recommend using a username and password that you deem appropriate to mitigate our design decision.
The problem here is that when something goes wrong, it's always the developer that is blamed. It's up to us to demand something better for ourselves.
A Challenge
This is a challenge to give privacy a bigger consideration in your application - in the same way you might be challenging yourself right now to use TDD or Agile. Shift away from the information arrogance that make us think we can ask people for their email account and password (with apologies to Jeff Atwood):
Never forget that people are after your data, all the time. I get reminded of this daily. All I have to do is turn on my FTP server for 60 seconds, and I get some great pings from people trying to hack my server:
I'd love to hear your thoughts on this. Are you willing to Kill Your Users Table?
Absolutley agree. Can't stand all the pages that wan't me to sign up. If it's not really important I try to use an account from bugmenot.
If I was starting a community today I would use OpenID for the authentication. You would still need user profile tables (but don't require users to populate it before accessing your site).
Is Windows Live ID ment to be used as an authentication mechanism like OpenID? I got the impression from Angus Logan and the Windows Live guys at TechEd that the Windows Live ID login control was ment for pages that used Windows Live Services like the chat control, contact sharing etc. but not as a replacement for authentication (like OpenID is).
@Jonas you might be right about the LiveID thing - I was thinking more of Passport; and the two are sort of intertwined.
I would agree as well. New sites that leverage OpenID make it super easy to join. For example, http://identi.ca/ an open source twitter like service offers OpenID as a signin method, which authenticates you via OpenID, then establishes your site profile afterwards. I don't have to remember login information, etc.
This benefits the user and the site. The user doesn't have to remember another username and password. The site doesn't have to worry about storing that information and keeping it safe.
This is fine for techie sites, but I can't imagine my mom signing up for an OpenID, or even understanding some site's description of why they use it and why they're sending her off-site if they want to sign up for their (seeming unrelated) site. Has anyone done any research to see how feasible something like OpenID is for the web at large? Or are we all clinging to some idea of a universal login that will never come to fruition?
Great points Doug. OpenID is probably the least user-friendly option out there. Why does no one ever discuss a middle ground between storing very little info and storing way too much? There has to be a practical solution for securely storing passwords that allows you to recover them.
@John S,
I agree there should be a balanced approach to storing the right amount of user data, but I think that balance should never include storing passwords in a recoverable format.
Is there really much complexity difference between being emailed your plain text password vs. being emailed a link to set a new password? I think the minimal difference is more than compensated by the huge security advantage.
>>>This is fine for techie sites, but I can't imagine my mom signing up for an OpenID<<<
The "mom" argument is completely invalid here. If your mom can understand what "registration" is, she's likely smart enough to figure out that we're asking her to do it somewhere else. This isn't rocket science and we need to take it seriously. WE are part of the problem when we disable security in the name of convenience (see spez's comments above).
Chances are your mom has an AOL account anyway, and if so she's got an Open ID.
>>>OpenID is probably the least user-friendly option out there<<<
Seriously? Do you have one? Have you tried it yet? I don't agree with this.
This is a challenge... free your mind. Users aren't stupid - they'll get it if we make them. Mom will understand.
This is a system you can use to make sure your users' information is encrypted.
Requires a password from the company aid the encryption.
Step 1) Encrypt the given username with the given password using SHA-512 (irreversible)
Step 2) Encrypt the given password with AES using the given username
Step 3) Encrypt the given email with AES using the company password
Features you keep:
1) Forgot Password: After submitted the email address and the username associated with the account, the password can be retrieved. Email address is confirmed after decryption, password can then be checked using the username submitted.
2) Subreddits: Subreddits for an account dont have to be encrypted but can be retrieved if the encrypted form of the username is saved for the session while keeping the origional username secret.
My 2 cents.
Does open ID allow you to store the users email for marketing purposes?
mythprogrammer: all you've done is made the username the password... now they have to remember 2 things, and you can't send them their password unless they remember the username.
>>>The "mom" argument is completely invalid here. If your mom can understand what "registration" is, she's likely smart enough to figure out that we're asking her to do it somewhere else.
Whilst I think some kind of OpenID solution is a good idea (ie some kind of net-wide login), I do actually agree with Doug Mayer that this IS another hurdle for the more technically challenged.
I am sure most "moms" can follow the link to OpenID, the question is will they remember who sent them there and how to get back to that location (unless OpenID stores this and provides the back link in a clear and obvious manner - I've not used OpenID). I think many of our clients would be too scared of loosing potential customers if we implemented OpenID only access.
I'm surprised that no one has brought up CardSpace(http://netfx3.com/content/WindowsCardspaceHome.aspx). I think CardSpace is supposed to be a proposed solution to this problem. It will hopefully provide a user friendly enough interface that my Mom can use it. I personally like OpenID better, so I can use it from anywhere and I don't have to worry about carrying my "Cards" with me or duplicating them on multiple computers, so I think there's room for both solutions.
The "mom argument" is indeen completely valid, you have no idea how "stupid" people are once they try using a computer. So it has to be a easy as possible for users to register, and directing them to another site does not count as easy for many people, believe me.
There is a great example of the approach that you suggest in this article implemented at "whoisi" (http://whoisi.com). This service let's you "follow" people without any registrations whatsoever; everything is stored in a cookie on your browser. If you want to change browsers, you just click on a "Login Later" button which generates a URL for you to visit on another computer/browser. I realize that this approach might have drawbacks (such as loosing the cookie, etc.) but i makes joining the community super fast.
PS: How about adding OpenID auth option for comments on your blog? :)
I think Ryan, and by extension Barry Dorans with CardSpace, are on the right track. Even using OpenId, LiveId, YahooId, whateverWebSiteId, still puts your user information somewhere on the intertubes. Why shouldn't that personal information stay with the one person it *should* stay with? You. Authentication should be done via impersonal tokens IMO.
But at that point, the sites lose any email/sms/IM information they have about you. Meaning the website can't send you and your network notifications except through the system (e.g. you have to log in to see your notifications). How would a user table-less system handle notifications?
I have to disagree with you on this one, Rob. The 'mom' argument is very valid. I've spent a lot of time teaching community internet courses through the years... and have also spent a good deal of time trying to teach various family members things as well. Non technical types sometimes are almost even proud of not understanding 'that computer stuff'. It's like a Badge of Non-Geekiness.
You expect that, I guess, from the general public, but the real shock came when I was in a position to teach university professors in a science department how to use an online course management system. The basic understanding just wasn't there... and many times they were very resistant to learning all but the very simplest things. If it was too complicated, they just basically 'shut down' and the new information didn't stick.
I think OpenID is an interesting idea... but I don't think I would use it on a general-access site at this time. Of course, providing it as an 'option' isn't a bad thing to do for the ones who want to try it.
As a user, this sounds great. But as a web developer, why would I want to give up my users table? It's one of the things that's giving my startup value. When people talk about how many quadrillion dollars Facebook is worth this week, it's not because Facebook has some awesome new scaling technology, or that we couldn't figure out email. It's because they have more user data than anybody else. I'm not seeing an incentive for us to do this, unless everybody else does. From an Adam Smith perspective: knowing less about our users does not make it easier to serve them.
And no, our mothers are not stupid -- nobody said they were. They just have better things to do with their time. My mom *could* learn FORTRAN IV, or figure out OpenID, but she has better things to do with her time. (She has no AOL or OpenID, either.) Again, it comes down to incentive.
OpenID is a geek dream. Don't get me wrong: it's a good dream. It's one of those things which we know we're all going to have to go to one day, because the fundamental technology is something users will eventually demand, like IPv6, but there's no killer feature that makes everybody want to go out and switch today, like there was for Netscape or Google.
I have to agree with the "mom" argument as well. The others have put in plenty of arguments why and I agree - there are alot of people who COULD understand, but won't bother to try.
On a technical note, how does this even help us? The personal information is still out there somewhere that a hacker or dumpster-diver could grab - but now they've got access to ALL of the sites you visit, not just the one they hacked.
What happened to Reddit was a physical security mishap - something that no technical wizardry is going to prevent.
Sure, this is nice for us tech types because we get single-signon, but sucks for the website's business - they've just traded convenience and valuable marketing information for the ILLUSION of security.
>>>Encrypt the given username with the given password using SHA-512 (irreversible)<<<
There is no such thing as "irreversible" encryption. Every cipher can be cracked - it just takes time.
>>>Does open ID allow you to store the users email for marketing purposes?<<<
It depends what the user is willing to share with you. If they allow you to see their email, then what you do with it is up to you.
>>>I am sure most "moms" can follow the link to OpenID, the question is will they remember who sent them there and how to get back to that location<<<
Some providers handle this redirection for you (like myopenid) once you sign up.
>>>I think many of our clients would be too scared of loosing potential customers if we implemented OpenID only access.<<<
Valid point. But the counter to that is that most people already have an Open ID so it's a matter of education.
>>>I'm surprised that no one has brought up CardSpace<<<
I'm going to do a Screencast with Barry Dorrans (MS Cardspace team) next week.
>>>you have no idea how "stupid" people are once they try using a computer.<<<
Gimme a break. People aren't stupid - websites are. The difficulty isn't your users (they can enter their information if they so choose and they know how to read). What people don't like is telling a website for the thousandth time their First/Last name.
>>>PS: How about adding OpenID auth option for comments on your blog? :)<<<
The Graffiti guys are on it now :). Just waiting patiently for the plugin :).
>>>How would a user table-less system handle notifications<<<
Agreed - this is a challenge. My thought here is an opt-in policy.
@Shannon: I was a trainer for 5 years before I was a contractor. I taught VB6 and know full-well the range of folks you get.
The missing element here is not "hey go get an Open ID" it's the bit of text on your site that asks a user "hey did you know you can log into our site with your AOL ID? Or Yahoo? Or Flickr? Chances are you have an Open ID and you don't need to keep your information with us. Just go to..." and that's it.
>>>But as a web developer, why would I want to give up my users table? It's one of the things that's giving my startup value<<<
No it isn't. Every startup thinks so - the value is the revenue model, not the user count (it's a common myth).
Facebook is worth money because it has a lot of users that use the site. If they went with Open ID (and kept their users) they'd still be worth a lot of money.
>>>I'm not seeing an incentive for us to do this, unless everybody else does<<<
And that's the problem. Get your data stolen once and perhaps that might be an incentive for you.
>>>knowing less about our users does not make it easier to serve them<<<
I didn't say that. There are lots of ways to track and "know" alot about your users. You can audit what they do on your site, sniff the IP and Geo information, collect all kinds of stats. You don't need their personal information for this, and you don't need to save a username/password.
>>>My mom *could* learn FORTRAN IV<<<
So could mine. Strawman.
>>>She has no AOL or OpenID, either.<<<
I'll bet you $5 USD that she, in fact, does.
>>>but there's no killer feature that makes everybody want to go out and switch today<<<
Market-speak. The pitch to a user that "we don't want to store your data" is a nice feature in and of itself.
>>>On a technical note, how does this even help us? The personal information is still out there somewhere that a h
>>>On a technical note, how does this even help us? The personal information is still out there somewhere that a hacker or dumpster-diver could grab - but now they've got access to ALL of the sites you visit, not just the one they hacked.<<<
This is a very good point and one of the issues with centralized security. The problem, however, is that it's monumentally easier to steal your information if it's all over the web, and ultimately easier to defend it it's in one place.
For instance I know, for certain, that my personal information has been stolen three times. Once from that massive credit card debacle, once from a stupid hacked up online store, and once from a site I won't name.
Whether it will be stolen from myopenid - I don't know. The thing I DO KNOW is that they salt/hash my password and it's unique, and I change it all the time. I feel better about that.
>>>What happened to Reddit was a physical security mishap - something that no technical wizardry is going to prevent. <<<
Not really. Yes, they literally lost the backup drive but on that backup drive was unencrypted password information. It's not wizardry on that score.
>>>they've just traded convenience and valuable marketing information for the ILLUSION of security.<<<
How does having a username/password/firstnam/email equate to valuable marketing information?
A lot of pushback here and that, in a way, is what I expected. YOU think that most users are stupid and I counter with SITES are stupid (most of the time). My mom can figure out Yahoo/Gmail/LiveMail. So can my wife and my older daughter, who's six.
If we present people with an alternative and educate them on how to do it, it serves us and them.
There is no loss of user information - you're just letting the authentication happen somewhere else. It's not that hard, and it's not that crazy.
It seems to me that everyones complaints boil down to sending people off site to sign up for OpenID, why can't we have our site implement some kind of proxy OpenID signup system? i.e. they enter all of their information into our site and then we ship it off to an OpenID provider.
The best way would probably be a OpenID widget that signs them up, that way our site does not have the potential to retain their OpenID login info, while at the same time having the potential to style the form any way we see fit.
I am in a business that provides a very customized experience for our customers based on information provided by their employers. I would still find a system like this very useful. To the people who find that users are too stupid to use a system like OpenID I have to completely disagree. I primarily deal with a customer-base of people over 65 and found myself singing the same tune, but boy did I under estimate the intelligence of our customers. I look forward to any features that make our customers experience easier for the long haul.
Additionally I get sick of hearing people say that all this information is so valuable. It isn't and I'm willing to bet that the majority of people that have this information don't use it, and if they do use it they are annoying their customers more than they are providing any service to them.
I like the opt-in idea for notifications. This way anything you send your customers it means you're providing a service that they have requested.
Also, I can only imagine moving to a system like OpenID would only help matters when needing to integrate a single sign-on experience with other companies. We are looking into this right now and every company we deal with has their own proprietary system that requires development effort that wouldn't necessarily be needed if everyone used OpenID.
OpenID is fundamentally flawed, and a phishing paradise. Please don't advocate using it. You can find a nice roundup here: idcorner.org/.../the-problems-wi
@Daniel I realize that ever solution has it's flaws - but everything in the article you mention is based on my creating a site to phish an OpenID.
It is a good point - but phishing is not confined to Open ID - it's everywhere.
""">>>[my mom] has no AOL or OpenID, either.<<<
I'll bet you $5 USD that she, in fact, does."""
I think I'd win that bet. I visited her a week and a half ago. She uses email (through a small no-name ISP) every couple of weeks, and she occasionally starts a web browser and types a word into Google (the default homepage, which somebody set up for her). I doubt she's signed up for anything online, ever. Unlike us geeks, she lives in the real world -- her personal and business relationships are built and maintained with a (POTS) phone, the U.S. Postal service, and her car.
Unless Volvo dealerships are handing out OpenIDs, I really don't know how she would have gotten one.
>>.The missing element here is not "hey go get an Open ID" it's the bit of text on your site that asks a user "hey did you know you can log into our site with your AOL ID? Or Yahoo? Or Flickr? Chances are you have an Open ID and you don't need to keep your information with us. Just go to..." and that's it.<<<
The question is then, what happens if they don't already have an OpenID? A quick random sampling of OpenID sites suggests that a lot of them use it as an additional authentication scheme, but also have their internal users table. It'd be interesting to know how many people opt to go create an OpenID account versus just typing in a new registration right on the page they are currently on already.
My assertion is just that people who are technically proficient can sometimes be surprisingly resistant to going out of their way when it comes to doing things outside of their field of expertise or arena of common knowledge. Sure, they aren't stupid and they could learn, but do they want to?
Having the OpenID option on a site is probably a good thing, I wouldn't argue that point, but killing your internal users table and forcing all users to use an external site for authentication is questionable just because it adds another layer of perceived difficulty to the registration process. Does that really facilitate your goal to... Sell Stuff(tm)?
That being said, if, as Justin notes, you can make the registration process unobtrusive and integrated into your site, then that'd be a moot point. However, considering that there are many OpenID providers out there that you could choose, I'm not sure that is feasible. You'd have to either force all your users to use one provider or you'd have to write code to handle every possible provider the user could choose.
I've been toying with the idea of implementing OpenID as well, so this discussion is great. I'm still not sold on the value, and I'm definitely not sold on the value of using it as the sole authentication service.
>>>and she occasionally starts a web browser and types a word into Google (the default homepage, which somebody set up for her). I doubt she's signed up for anything online, ever.<<<
Then I spose I have to ask why we're using your mom as a test subject here?
@Rob Conery
>>>Encrypt the given username with the given password using SHA-512 (irreversible)<<<
>There is no such thing as "irreversible" encryption. Every cipher can be cracked - it just takes time.
A couple of things to clear up here. While Mythprogrammer's idea is horribly naive, he is exactly right that SHA-512 hashing is irreversible. *All* one-way hash functions, even MD5, are irreversible. Hashing is not encryption and, by definition, a one-way hash does not contain the original data. So there's nothing to "reverse". iow, a hash code is not ciphertext that can be decrypted into plain text. Of course, you can come up with, perhaps different, plaintext that produces the same hash code as a hash code that you've captured. But that's called a collision. And while the ability to quickly produce collisions for a given hash code is what ultimately renders a hashing algorithm "insecure", the hash code still does not actually contain the original plaintext and nothing has been "reversed".
On the other hand, all forms of encryption are, by definition, reversible. Otherwise they wouldn't be of much use.
Hi Rob, I agree to some of your points. It always came to me that implementing authentication / authorization for each application is re inventing the flat tire each time. It is very prone to errors, bugs, security leaks.
And for particular NotNorthwind case, I think both should exist there as people are still doing introductionary(is that a word?) demos with implementing authorization on northwind to show how they pull data from db.
HOWEVER,
The only places that we use usernames and passwords are not community sites, and even it is an important issue, sorry but it is not the *ONLY* issue.
- How about performance ?
- The complexity it adds to the process ? I have yet another dependency on a third party service to integrate in my test for e.g
- High availability ? How can I trust OpenID - or any other provider to be online and working all the time - since my site will not be usable at all at that time too ? (Just think of when twitter is closed, the appearance on your badge in your web page at that time)
- Scalability ?
- Offline web sites and intranets with no access to internet? Ok, just gone too far :) But you got what I mean :)
Other than these, I see value in a well recognized authorization service, but with current state we have with the authentication / authorization providers I don't think that they are that mature enough (yet).
Rob: People ARE stupid when it comes to computers, and most people do not have OpenID as you said. You must get out of your geek-circle and realize what kind of people are browsing the web. I have seen doctors and professors with master degrees in you-name-it and they could not figure out how follow a link on a website.
@Ryan - thanks good points :). I saw "Encrypt" and I was off. You are very correct and thanks :)
@Sidar - good thoughts as well. A downside to the "programmable web" yes?
@IceMan: I don't live in your world, and I don't consider users stupid - call it a "quirk" of mine. As far as the rest of your comment - whatever.
Having looked into OpenID more (never used it myself) I have to say that it does look like the potential for phishing attacks outweighs the benefits, and there is no way to fix that problem so far as I can see.
@Justin: I'm hearing about that a lot and I do agree having seen what's happening with PayPal and eBay. Continuing the pile up of user data accross the web (to me) doesn't seem to be an answer though.
I'm really going to challenge you guys now. I conjecture that it's possible to design websites and web applications, with user-specific functionality, without ANY kind of identity management at all. This means no users, user profiles, and no signing in of ANY kind.
Any features that seem to depend on user-specific information can be accomplished via other means. Ultimately, you end up with a design that is more compositional, and which can be used to delegate authority easily (unlike current designs in which no delegation is possible).
See the <a href="http://waterken.sourceforge.net/">Waterken web-keys framework</a>. It's essentially REST + security = distributed web objects. Very compelling.
In fact, I'd wager you've all used or built something similar to this at one point, you just never took it to its logical conclusion like Waterken does. For instance, inviting someone to view a google web album, or signing up for a which provides you with a one-time unguessable URLs used to activate your subscription.
Rob, I don't think anybody is saying that you are setting up a honeypot by using OpenID. The problem is that by promoting OpenID it's easy for a seemingly innocent site to be a honeypot and that's the problem with OpenID. The same analogy would be using the same key for your house, your car... your work place... imagine losing that key or imagine that some of the lock can take an imprint of your key and the owner now have access to all of your belonging. That's the fundamental problem with OpenID and it's a scary thing.
On the subject of systems like OpenID I can only see the phishing problem being fixed by an agreed upon open standard for all of the major web browsers (which seems pretty unlikely). A system in which the browser can request a temporary sign-in key for a website and the third party website just passes that key on for authentication.
Since we are specifically talking about an e-commerce site, I think we should probably discuss credit-card storage. I'm assuming that you don't want to store users credit cards, and therefore don't need to store _any_ user information whatsoever (at least long-term). If that's not the case then you already have the issue of storing their credit-card info in a secure manner, and at that point you'll need most of their relevant information anyway. So my question is, what is your proposal for doing check-outs without attaining their information?
Rob, Secondly I also agree with you I don't think user are stupid. However users are gullible and that including you and me and the rest of them. That's just human nature. I see the problem with OpenID right now because I have the security hat on. But other day (if OpenID become a common thing) I might be tricked into logging into a phishing site with my OpenID and not think about it.
@Firefly/@Justin: Phishing is a big deal, to be sure and the only way to combat it is with user education. PayPal, eBay... those guys are propelling a lot of what you're asking for. Browsers are getting smarter... but I agree - it's a problem.
I think we disagree on how much so.
>>>If that's not the case then you already have the issue of storing their credit-card info in a secure manner<<<
I do indeed, and that's a sign that hangs on my PC that says "NEVER, EVER". I don't care what the client demands, because if something happens, it's my fault right?
>>>So my question is, what is your proposal for doing check-outs without attaining their information?<<<
You, sir, just won yourself a spot on the next screencast for the store :). I have many ideas...
I do agree with you on one point: having the same password for two different websites is bad. It's one of the worst mistake one can make. I don't know much about OpenID, but I have the feeling that the system promotes exactly that: one password for a lot of websites.
Besides, it seems to me that having one identity for several websites promotes data mining. And seeing AOL and Orange in the provider list, does nothing to contradict that idea.
Passport/LiveID is no better, since it's a Microsoft service. Yes, Microsoft, the spyware editor. The company which has basically no clue about security. Do you really want to trust them with your data? I sure don't.
>>>Microsoft, the spyware editor. The company which has basically no clue about security. Do you really want to trust them with your data? I sure don't.<<<
...says the guy writing this comment using Vista and who's email is stored in Outlook. MS is evil blah blah blah talk to me when you drop the OS back off at CompUSA.
>>>I don't know much about OpenID, but I have the feeling that the system promotes exactly that: one password for a lot of websites.<<<
It's an auth service - you login with a site you trust and it verifies that you are who you say you are - that's all. It's like a personal SSL cert.
Rob, are you not afraid of Big Brother !? Imagine how easy to make a profile of us, just checking our browser habits.
Jon Galloway touched it in your screencast, but why should we facilitate for the state or big companies to check us up ?
Lela that's a good point :). Consider the whole YouTube thing right now - they're being ordered to turn over ALL of their User data - this means all of the video's you've ever watched (assuming you have an account there).
Think of THAT profile :).
I agree with using something similar to OpenID, allthough it was very, very difficult to register. Many attempts failed because of funny or invalid redirects.
But most importantly; not everyone reading the blog is from the US (AOL). We do have internet in Europe as well.
Numbers do kill, but ratio does count for something: www.nationmaster.com/.../int_use_percap-
@Daniel Serodio:
arguments about OpenID are fundamentally flawed. OpenID does not in any way require to use password authentication. It just "outsources" authentication to designated authority. You can use SSL certificates or other methods with OpenID, which are more phishing-proof than alternative of not using OpenID at all.
Rob : I assure you I looked only once (or twice) at that film. How did you know ? :). Imagine if BB just with one click can check everything and choose what is relevant or not. Not all eggs in the same basket !
Phishing, Identity Theft, etc. Why can't people you do these sort of things use their knowledge for good. I can't stand people who try to hack into my servers and I think everytime: "Why don't you use your knowledge for good like finding a way to cure cancer." (A little far fetched I know). I think of these people as lazy and good for nothing. They are trying to get away with doing as little as possible and having as much as possible without regard to what frustration it causes people. Why don't they get a real job and stop trying to impress their friends with what they hacked into.
IMO, OpenID is far more susceptible to phishing attempts because users *expect* to be redirected to a different site for authentication; most users aren't going to pay attention to the URL in the browser when they get to a familiar looking site.
I agree, however, with the basic premise of OpenID but I think there needs to be more done on the provider's side of the house to prevent phishing. I know Bank of America and other banks display a Site Key (a random image the user picks when registering), known only to the user, so they know they're on the authentic site when they're authenticating.
Just my 2 cents...
I love the idea - and often wonder why my clients require so MUCH information from their potential clients.
I agree with the poster MUCH further up that said people will try NOT to learn/know and wear it as a badge of honor. (Like me and Degrassi on Sunday mornings.... shhhh...)
As far as making something easier - You try to make something idiot-proof - and someone morphs into a bigger idiot. Why else would Pop-tarts* need toasting instructions!! McDonalds' Coffee is HOT!!! The items in the overhead compartments may have SHIFTED during flight! Your beer didn't turn out because you didn't SANITIZE!!
*The author of this post is in no way affiliated with the Pop-Tarts brand, doesn't receive anything free from McDonalds' except napkins - and was given ZERO free miles. But was definitely knee deep in homebrew!!!
I did kill the users table on production once.
The customer was _not_ amused.
@Ayende - Ha! Awesome, I once did the same but just to the password field, let me tell you how secure your system is when everyone's password is just set to gibberish...
@Rob,
I appreciate the boldness of making a suggestion like this, but I take issue with your assumption that everyone who disagrees with you is knee-jerking and hasn't thought through the issue.
"'Not hashing was a design decision'...This design decision wasn't thought out all the way. The ramifications clearly not fully understood."
How do you know? All design decisions are tradeoffs. After evaluating the risks and rewards, the Reddit team made a decision. The fact that it was not the decision you or I would have made does not mean that they did not think it through.
"Billing records are kept at the payment processor, and shipping is likely kept in a waybill somewhere - both accessible by your application when needed."
Most payment processors REQUIRE name and address as part of the verification process, or at the very least charge lower fees when they are provided (since they are now at less risk). This is not something that you can just decide not to keep.
"The 'mom' argument is completely invalid here."
No, its really not...
"People aren't stupid - websites are."
Users don't have to be stupid to be completely unable to navigate the internet. My own mother has a bachelor's degree in journalism from Northwestern University. She's no dummy. But the moment she gets on the computer, and especially when she is on the internet, she is lost. She has no concept of how it works, or what she should do or shouldn't do, or what the dangers are, or how to avoid them. She's no hermit either; we have had a computer in our house my entire life, and my mother is on it all the time, using email, reading news sites and articles, and paying bills. But the moment anything unexpected happens, or she has to do anything more complicated than click on a link or a button...forget it. Certainly the development community could do a better job of creating user-friendly websites, but the fact remains that the whole internet phenomenon is still very new; we (the geeks) may have grokked it a long time ago, but most of the world still doesn't really get it, and it will be a long time before they do.
">>>She has no AOL or OpenID, either.<<<
I'll bet you $5 USD that she, in fact, does."
Mine doesn't, but more to the point, neither do I. I am a professional developer, I spend half my waking hours or more on the internet, and I don't have an OpenID account. At least, I have been through the list of sites on OpenID.net, and I don't use any of them. Maybe I actually have an account and don't know it, but then, what's the point?. Your unsubstantiated assertion that nearly everyone must have an OpenID account by now could be a reason why you think this process would be so easy; I submit (without any proof of course) that you are wrong.
All of this is totally leaving aside the issue that once you move to a single method of identification and authentication, a single compromised password opens up an entire universe of accounts to the hacker. Isn't that why we are always told to use different passwords for different sites? Why then is it a good idea to start using the exact same set of credentials everywhere we go?
My man David! I was waiting for the bomb from you (or Stephen) :). Cheers! Good points - let's hash it out.
>>>How do you know? All design decisions are tradeoffs. After evaluating the risks and rewards, the Reddit team made a decision. The fact that it was not the decision you or I would have made does not mean that they did not think it through.<<<
Dude - clear text storage of passwords is wrong, and dumb. No avoiding that one.
>>>Most payment processors REQUIRE name and address as part of the verification process, or at the very least charge lower fees when they are provided (since they are now at less risk). This is not something that you can just decide not to keep.<<<
Yes - they require when the authorization goes through, but you are not required to keep it. You have the authorization - give me a reason you need to keep the billing address?
>>>"The 'mom' argument is completely invalid here."
No, its really not...<<<
It's completely invalid. User's aren't stupid. They just don't get you and think like you.
>>>But the moment she gets on the computer, and especially when she is on the internet, she is lost...my mother is on it all the time, using email, reading news sites and articles, and paying bills<<<
This paragraph is really confusing. If your mom can pay bills online she can, with a little help from us, use her AOL/Yahoo account to log into our site. The assumption you're making here (which is unfair) is that I'm suggesting we "throw a button and textbox on a page" - no. We explain things - just like we explain "You're now going to pay me using PayPal... here we go!".
>>>I am a professional developer, I spend half my waking hours or more on the internet, and I don't have an OpenID account<<<
Really? No flickr? No Yahoo mail? Wordpress? Technorati? Fair enough.
>>>I submit (without any proof of course) that you are wrong.<<<
Interesting - it seems you love to assume I'm wrong a lot :). Either way dude it takes minutes to sign up.
>>>All of this is totally leaving aside the issue that once you move to a single method of identification and authentication, a single compromised password opens up an entire universe of accounts to the hacker. Isn't that why we are always told to use different passwords for different sites? Why then is it a good idea to start using the exact same set of credentials everywhere we go?<<<
Good point here and I'm thinking I should write a post on just this. OH! Hey! I did already - scroll up and have a read and this time, if possible, you could open your mind to the idea rather than (typicall) looking for ways to disagree with me :p.
In summary form: your ID info is more likely to be stolen if it's in more places. What is happening right now (identity info all over the place) isn't working - as i said above, my stuff has been stolen 3 times and I'm willing to bet (here I go again...) that yours has as well.
Poor Moms. Doesn't anyone have a dumb Dad?
But seriously (expect rare cases) it's not a matter of intelligence. It a matter of desire. Some users simple don't want to learn whatever it is you think is important for them to learn. I had a IT/programming job at a major university and I was amazed daily at internationally renown professors who were complete imbeciles when dealing with the most basic computer tasks.
Basically it boiled down to this: they didn't want to learn --they had more important things to do...But if you can convince them that there's a solution to something that causes them pain or something that will save them time -- that once imbecile will learn in seconds...
>> Yes - they require when the authorization goes through,
>> but you are not required to keep it. You have the
>> authorization - give me a reason you need to keep the
>> billing address?
Uhh... Maybe so the user doesn't have to enter it again when they make a future purchase?
I'm all for not storing credit card numbers in your database, but letting the user save their shipping / billing addresses for future use is generally considered (by me anyway) a nice feature. I know I appreciate when Amazon does it. In most cases it's the same as the shipping address anyway (which I'd wager you most definitely DO need to store for your shipping guys to use).
Some user info you really should keep or you're killing usability. And you can't just assume "well, I can get that from OpenID" because you can't guarantee all providers are going to store all of the information you might need.
Also, I typically like to get an e-mail notification when my stuff ships - so there's a good reason to store an e-mail address.
It all comes down to security versus functionality and I just don't think the security benefit outweighs the loss of convenience to the end user.
Now I'm not saying it might not be a good idea to let OpenID take care of authentication - but that's just username and password - it's a far cry from saying "kill your user table". You still need some sort of "user" table for personalization and that table will need a "username" column to match it to the authenticated user.
So, really, maybe we're not talking about "kill your user table" so much as "kill your password field"?
@Jamie: I do agree with you in terms of eCommerce. An "address book" is very handy indeed - but again I like the opt-in model (same for any email notifications). This is a far cry from username/password (which is the User table reference) and yes, it's a convenience and they are good.
But I say "ask, don't assume".
I am not quite sure if I have ever read a blog with so much angst in the comments day in and day out hah. My 2 cents: my mom is an internet retard (getting better, but still a retard) and would fall prey to phishing easily (but phishing isn't something technology can fully protect us from as it is a social engineering exploitation, so really doesn't matter in this talk) but also doesn't save N username/password combinations. I would imagine most "mom users" are the same way, and have 1-3 combinations they use (I remember seeing a stat that said 2/3 of all internet users use the same password for everything, this page states similar http://www.readwriteweb.com/archives/majority_use_same_password.php). So the "all your eggs in one basket" issue is present when using any authentication, except when not using a service like OpenID, you actually have the same basket of eggs sitting in many many places, including that basket of eggs that is sitting on the front porch of Mr. Reddit's (don't want to single them out, hell I have seen much worse things in the code that my company has) house with a big "here are some valuable eggs and no one watching them" sign on it.
I don't think Rob was trying to endorse OpenID as the answer, but more like a starting point for coming up with a better solution. I see issues with OpenID too (phishing mainly, but that happens anyways so it isn't like OpenID is introducing a security hole and keeping status quo isn't getting rid of phishing anytime soon), but I am also not going to count it out based on crap (http://idcorner.org/2007/08/22/the-problems-with-openid/ read this, most of it are flaws that exist without openid or just garbage). Bottom line to me is that something like OpenID solves at least one problem we have today, which is that our personal information is stored everywhere and the security of that data is VERY VARYING. Does it solve phishing? No, but does the alternative? No.
"...it seems you love to assume I'm wrong a lot"
True, but don't take it personally. I tend to assume that everyone is wrong most of the time. Even me ;)
"You have the authorization - give me a reason you need to keep the billing address?"
As already stated by Jamie...so that they don't have to enter it again the next time. I can't imagine that you think there aren't many users who would complain loudly if they had to re-enter their address every time they used the same site.
"If your mom can pay bills online she can, with a little help from us, use her AOL/Yahoo account to log into our site."
"We explain things - just like we explain 'You're now going to pay me using PayPal... here we go!'."
If I didn't know better, I would think that you have never had the pleasure of getting phone calls from family members who didn't understand why they were being redirected to PayPal when they bought something on a site that (as far as they know) has nothing to do with PayPal. Or getting repeated phone calls from those SAME family members about the SAME ISSUE after you had already explained it to them. In fact, from the general tone of your argument, it is almost as if you have never had to do any kind of end user support at all, which I find very hard to believe. People don't know how to use computers. They don't know how to use the internet. They just don't. And wishing they did won't make it so.
Again, I am NOT saying that users are stupid. But you don't have to be stupid to be technologically incompetent. I see it every single day.
By the way, I am not saying that "the Mom argument" is reason enough to not even consider using service authentication like OpenID. I am responding to your complete dismissal of the issue. Identity management and authentication are hard issues for professional developers to understand, and you think you can just "explain" to the end user that you have access to their Yahoo account? You don't think people are going to be confused by that?
"I'm thinking I should write a post on just this. OH! Hey! I did already..."
I don't see anywhere where you address the inherent danger in putting all your eggs in one basket. You simply assume that danger is more acceptable than independent accounts that are more susceptible to being compromised individually (assuming that individual sites have poor security, which we know they do), but have no relation to each other and therefore can't compromise each other. I am not convinced either way, and you have said nothing to try to convince me. A separate post on that issue would indeed be an interesting read.
"my stuff has been stolen 3 times and I'm willing to bet (here I go again...) that yours has as well"
I suppose its possible that some portion of my identity or some kind of account information somewhere has been stolen and I don't know about it. But there certainly has been no evidence of that (drained bank accounts, bad credit report, police showing up at my house, etc). Not to say that it can't happen; I know its possible. I'm sorry it happened to you. But a knee-jerk reaction of "no one is allowed to store user information anymore" doesn't seem like the right response.
@David
"I don't see anywhere where you address the inherent danger in putting all your eggs in one basket. You simply assume that danger is more acceptable than independent accounts that are more susceptible to being compromised individually (assuming that individual sites have poor security, which we know they do), but have no relation to each other and therefore can't compromise each other. I am not convinced either way, and you have said nothing to try to convince me. A separate post on that issue would indeed be an interesting read."
You should read my comment above, if 61% of users have the same username/password combos for all their accounts (and probably more use 2 or 3) then you already have the "eggs in one basket" issue in an even more dire situation, where your information is only as secure as the weakest place that stores it. And Rob has already ceded that storing contact information is more of a business decision if there is functionality that needs it in your app and that when he was referring to "killing the user table" he was more meaning login credentials.
So lets talk about the part about dumb users, I love arguments that to me feel like the religion debate.
>> Why can't we use a model like Paypal, or Kayak, or my electic company for telephone payments (they send me to a 3rd party to do the payment, the 3rd party gives me a reciept number, which i then have to tell to my electric company), it seems to have not hindered Paypal from catching on (are your family members startign to get Paypal by now)?
>> Because they are dumb
>> What about it won't they get
>> They are dumb
>> How can we help them get it
>> THEY ARE DUMB!!!
This could go on and on, and you will just keep saying the same thing back so why bother... except for the fact that your argument is weak because at one point a few years ago pretty much everything that exists on the web/computers today would have been "too hard for the dumb people to get". Some people might not immediately understand it, but isnt that the way pretty much everything in computers started?
I don't understand how we've come to the conclusion that OpenID is anything other than a phisher's dream...
Currently a phishing site has to look like your bank's site (or not depending on how stupid the user is) in order to get your login details from you. With OpenID, a phishing site can be absolutely anything - social networking site, blog comments, etc. - the only thing that they need to do is have the following:
Login using your OpenID:
OpenID:
Password:
If people are used to using this single login on every site that they go to, why are they even going to think twice about doing it on yet another site?
Because people are stupid, this is what they will do. The rampant success of phishing is surely enough proof of this.
You say that people aren't stupid, they just need educating; to me stupid and uneducated are not too far from being the same thing.
I would rather put it on myself to remember different logins for many sites, knowing that if that site gets compromised and my login details go awol someone only has access to that one site - this is far better than the one time that I get duped into using a spoofed OpenID login form and give away access to my entire online identity.
Obviously this is assuming that I am going to avoid password re-use etc. Let's face it, if I wrote down all my login details then Joe Hax0r Bloggs isn't going to break into my house to steal this list(s) of login credentials is he now? Fred Housebreaker Smith is also going to be more interested in my 42" plasma, laptop etc. than this list?
@Carl - I don't think anyone is saying OpenID isn't a place for phishers, but given the fact that ~60% of internet users use the same password for everything, those 60%'s identity is only as safe as the weakest site. What if OpenID got the browsers involved and OpenID was built into the browser, or rather than the site hosting a form, they had to be directed to an official OpenID login page with a sitekey or a security question(s) that the user sets up at the OpenID site? Don't say "well OpenID in its current form doesn't do what I want so screw it". Lets improve it or move to something better.
Also, I know you said that you are assuming the avoidance of password re-use, that doesn't happen though. And who only needs their passwords while they are at home? So you think someone who steals your wallet (lets just say Joe Dumboldguyuser keeps that list in his wallet) wouldn't try and see what sites he could gain benefit from if he found that list? But I still keep thinking "is OpenID claiming to fix this issue?" How does a thief ignoring your list of passwords versus your sticky with your one password have to do with this? The point of this talk was to focus on the issue of storing login information, so lets talk about that. What if OpenID required you to be directed to their site, they had site keys and security questions and if the browsers would display a picture of Buddy Christ giving that thumbs up/point thing if the browser validated that this was a true OpenID form?
just saw that in the 2nd paragraph I meant to say "And who only needs their passwords only while they are at home?"
@Kevin
In fairness, I think that we can probably rule out the browsers as they seem to avoid adherance to standards as much as possible. However, I do think that this would possibly be the best way of keeping OpenID secure, as you say. I don't think that redirection to the openID site is a very good thing to do, I personally know that being sent to the Mastercard/Visa Securecode site when I purchase items from certain sites annoys the hell out of me - though I would be more reluctant to use it if it was an embedded form on the site.
With respect to Joe Dumboldguyuser keeping his password list in his wallet, this will be the same guy with his PIN post-it note'd to his bank card and who is probably going to be pretty lax in general security-wise and will probably also be a phishers dream.
Your comment about not only requiring passwords at home is a fair one, but I personally only use a few logins when I am away from home - these are all passworded differently and I don't have a problem remembering them.
'~60% of internet users use the same password for everything'
So we need to force ~40% of internet users to use the same login for every site because ~60% of users do this anyway, even though this is considered bad practice - it just seems to me that we're throwing the baby out with the bath water here?
'What if OpenID required you to be directed to their site, they had site keys and security questions and if the browsers would display a picture of Buddy Christ giving that thumbs up/point thing if the browser validated that this was a true OpenID form?'
If I understand this correctly then this is suggesting that every login to an OpenID enabled site would be similar to how we currently log into bank sites - this seems a bit overkill to me, but yes, it would be more secure.
I guess I really just have a problem with putting all my eggs in one basket as far as security goes.
@Carl
Is there a restriction on having only one OpenID? How about you continue to make a new account for each site using OpenID, then you have your cake and eat it too. This is assuming OpenID is storing it in a secure manner, but I bet it is and then there is no doubt that your info is much safer than if stored in companywhomakesdumbdecisionsaboutsecurity's db (again, not from phishing, but phishing is a problem nonetheless, with or without OpenID or an OpenID like solution.).
I would rather have my eggs in a safe basket.
So what if OpenID still stored your personal information, but on a site by site basis. When the site needed to validate or update, it did so through some services?
My main concern is not knowing what the hell security these sites have on the information I am giving them, and as a developer it would make my life easier if all that security nonsense could be handled for me hah.
Hmmm, I like your thinking. Moving away from the 'single sign-on' aspect and more towards the 'secure storage of information' end of things.
That would definately make me happier (all I want is to be happy, honest) - wouldn't work too well if you had to use a different email address for each one, however I guess I could use the gmail carl+blah@gmail.com mechanism and this would have the added benefit of then allowing me to use labels and things to organise mails from each site (if I could be bothered!)
Like you say, this makes no change to the phishing situation, but the purpose of OpenID is not to address this issue anyway.
In my case, the issue of how much you trust Google/gmail is there but you're pretty much always trusting someone else for your security anyway...
http://google.com - google