ASP.NET MVC: MVC Storefront, Part 4 - Linq To Sql Spike

Nine Inch Nails? They rock.

Great stuff, Rob. Amazing how easy it was to map the LINQ to SQL classes to your own and have the "where" clause pass through like that.

I guess if your classes had relationships (like a Product.Category property rather than Product.CategoryID) it would be a little more complex, but "connecting" instances would be the job of the service class, right? Similar to the way your GetCategories() service method takes a flat list of categories and turns it into a tree?

I watched all 4 videos just now and it's quite good stuff. Maybe slow down a little and explain some of the TDD concepts a little more at some point? Being a noob to TDD, I was confused when you began implementing the test repository early on, it wasn't immediately clear that you were doing that just to isolate some of the method tests as part of TDD.

Then later when you did the sql repository with an integration test, I missed the part where you delineated what makes a test an 'integration' test or a 'unit' test. (Presumably, the crossing of a layer/tier within the test?). Do both get equal treatment under TDD?

$0.02

Hi Ray - an integration test is pretty much shorthand for "hitting the database" :). It applies to integrating your code with another system - and in the land of TDD that's equal to talking to a DB.

RE spending more time - believe me, I want to :). The hard part is focusing on the main point here which eCommerce with MVC. Can you tell me a bit more about what you'd like to see?

I'm keen to see how the use of LINQ to SQL and IQueryable<T> ties in with unit testing - particularly in terms of mocking the repository and making sure its methods meet your expectations. Would that even be possible? Could your test actually expect a certain combination of .Select() and .Where() calls?

That looks great so far! You've made 'reading' data look pretty easy but what about 'writing' data?

Having been fighting with LINQ to SQL a lot recently, trying out context-per-request and context-per-aggregate-root and the replay model and whatever else I can think of to try and make it play nicely with a DDD-style domain model, with limited success, I'm very interested to see what strategy you go with.

Hi Rob,

Interesting set of videos. My one main question about this is why are you wrapping Linq to SQL objects with your own objects? Why not just use the Linq objects?

When you come to updating and inserting data, won't you have to do the conversion again from your object to Linq's object?

It just feels a bit unnecessary. Straight off, I also found it harder to read - I initially thought Category was the Linq object. So how different are your actual objects?

Ben

Blog.BenHall.me.uk

@Matt Hamilton

Using a tool such as TypeMock, you would be able to do this as this has the ability to mock Linq at a much lower level and verify the correct Linq query was executed.

At the end of my post on mocking Linq (I think there could be better solutions to this now) I mentioned TypeMock, at the time there support wasn't great but it has improved over the past few releases.

blog.benhall.me.uk/.../how-to-unit-tes

What I guess Rob will do is mock the ICategoryRepository which his DAL implemented in his Services layer and not be concerned with the actual Linq implementation. Similar to my post.

Nice videos.

Can you post the source code after each video?

Also I would like to hear some more on why you picked linq vs. nhibernate (maybe castle's activerecord) for example?

I wander how easy it is to refactor your wrapper objects after you change some table definitions.

Unrelated question - sorry but I can't figure it out!

How do you get those Avatars to appear near you name in comments? There is not registration or anything here!

AHHH !

Rob,

Thank you so much for this particular post. i'm not even finished yet and have to stop to tell you how much i appreciate it.

Watching you drive this spike through your code answers a few questions i've been struggling with about exactly this: how can i use Linq2Sql without littering my model (and worse, my presentation layer) with SqlMetal-generated classes.

At 11 minutes in when you show how your SqlRepository code maps the IQueryable<SqlMetal-Category> to an IQuerable<Model-Category> __without_firing_the_query__, a light went on.

I'm one of those folks who "didn't even know you could do that" and i want to thank you for showing off this particular language feature as it removes a block in my own project that i've been stuck on for several weeks.

As with you, Linq2Sql and IQueryable work fine for me at the moment, provided that i can separate them to isolate my database dependency and make change less painful down the road.

The last few minutes of this episode are worth the wait and the entire effort.

Looking forward to watching your ongoing learning curve, and thanks so much for sharing.

It's interesting to watch someone else go through a real world scenario of keeping application models seperated from the storage model..

I wondered about what model would be best for a orm in this case, I looked at concepts of the ORM being able to ask the ORM to map directly to your classes (without markup on your classes), by default - the orm would build up a runtime mapper (if not provided) which used reflection to determine the best guess at the format of the database..

Given that this doesn't play well, then you would provide 'mapping hints', not as meta data on the target classes, but as a collection of objects in the underlying call to in the ORM..

This way you stand the data off the model classes - however, I'm sure theres tons of issues regarding how efficient said design would be..

@Matt:

>>>I'm keen to see how the use of LINQ to SQL and IQueryable<T> ties in with unit testing<<<

In part 1 I stubbed out the TestCatalogRepository and generated some test data - this is how it ties in (and why I did it this way). Yes - everything you're asking about in this question is currently dealt with in a "fake" repository that implements ICatalogRepository.

The reason I went with IQueryable is so I don't need to implement a ton of methods on ICatalogRepository - right now there are only 5, and creating a new one is very easy.

@Ben:

>>>My one main question about this is why are you wrapping Linq to SQL objects with your own objects? Why not just use the Linq objects?<<<

I talk about this some in the video - I want the DB structure to be completely separate from the Model. I know there's overlap (right now) but I don't want the DB to push the architecture of my model, and vise-versa (in other words, I don't need a Culture object in my model).

>>>When you come to updating and inserting data, won't you have to do the conversion again from your object to Linq's object?<<<

Yes - this is Right/Left coding and it sucks :). But I'm clever and I might just write a utility to handle this for me :).

>>>It just feels a bit unnecessary. Straight off, I also found it harder to read<<<

I can dig it - hopefully when I implement an AmazonRepository you'll see why I did it this way :).

>>>So how different are your actual objects?<<<

When you see the ShoppingCart, you'll know why I did this.

>>>What I guess Rob will do is mock the ICategoryRepository which his DAL implemented in his Services layer and not be concerned with the actual Linq implementation.<<<

This is all in part one, with the TestCatalogRepository. I don't mock it, per se, I stub it with a fake data source.

@drozzy:

>>>Can you post the source code after each video?<<<

Oops! - meant to :). Added the link above.

>>>Also I would like to hear some more on why you picked linq vs. nhibernate<<<

Simplicity, for right now. Many more people are used to linqtosql and it's footprint is rather small compared to NHib. Also, NHib's not ready yet with Linq To NHibernate. I might move there in the future.

>>>I wander how easy it is to refactor your wrapper objects after you change some table definitions.<<<

Right now it's a matter of implementing 5 methods - it's why I went with IQueryable :).

>>>How do you get those Avatars to appear near you name in comments<<<

Magic :). I'm a Gnome Mage. That and Gravatar :).

@David Alpert:

>>>I'm one of those folks who "didn't even know you could do that"<<<

Me too until I remembered I did this a while back :).

@Stephen:

>>>It's interesting to watch someone else go through a real world scenario of keeping application models seperated from the storage model..<<<

Yah, we'll see how this works out :).

Thanks for clearing up some this, I look forward to seeing how it develops.

I take it you would do create your own objects for SubSonic objects as well? Even with the new Repository style?

I'm with Ben and am eagerly waiting to see the purpose of abstracting away the LinqToSql Models. It just seems to me that you give up a lot of the benefit of delayed execution in linq when you go this route, especially when I start thinking of all the dynamic filters and what not that you'd likely use in a commerce app.

@Justin I don't think to lose anything in terms of delayed execution when you go this route, you are just returning a different object to the caller, so I think it works the same in that context. For me, it's more about the maintainability and readability and if it is even required. I do know a few people who have separated there model, I think they use xml to do the mappings between objects and schema. I have done this before with SubSonic because of the ActiveRecord style, I haven't done it with Linq tho.

I do feel that later on, the design could demand doing something like this so I'm sure Rob is right.

Saying that, generally speaking (one of those purist moments I think) shouldn't the TDD process have encouraged you to separate the linq model from your conceptual when you needed to (via refactorings) - instead of you doing this upfront design because you are going to need it later?

I comes down to how much up front design you feel is required, Martin Flower (I think) has made the point that you know your going to need a database - so why wait until TDD demands that you to need one?

@Ben:

>>>I take it you would do create your own objects for SubSonic objects as well? Even with the new Repository style?<<<

I'd probably make SubSonic objects IQueryable :). In fact... hmmm...

@Justin:

>>>I'm with Ben and am eagerly waiting to see the purpose of abstracting away the LinqToSql Models. It just seems to me that you give up a lot of the benefit of delayed execution in linq<<<

I'm keeping the delayed execution, just adding some right/left mapping code to use my model. It's hard to grok at the moment, but when I make more repositories (like MySQL, etc) maybe it will make more sense.

@Ben:

>>>shouldn't the TDD process have encouraged you to separate the linq model from your conceptual when you needed to (via refactorings) - instead of you doing this upfront design because you are going to need it later?<<<

a Spike is more of an Agile thing. The question came up many times RE how I was going to handle my data-access, and would it support my repository. So I exited TDD mode and spiked it.

Oh yes, you did say - sorry :)

Good demo

but it seems to me a ideal study case and not a practical case

I am impatient to see how you deal with update, insert, etc and unit of work concerns (change tracking) and too, how you deal with relations between objet from the domain

In your example, you do not use linq 2 sql entities capacities about unit of work and relations, you wrap with your domain objects and, by me, if it's the good solution, i feel it frustrating to do not use what linq 2 sql can bring us (even if it's not enough).

So how will you deal with the domain driven design concerns or do you propose just a service layer with ad-hoc functions ?

Thanks a lot

Sergei

Wrapping those objects over LINQ to SQL like that is throwing away 99% of what it provides - update change tracking, relationship navigation, identity mapping, relationships between new objects, update sequence etc.

I get a feeling you're going to find life more difficult going forward as you reinvent some of this.

LINQ to SQL provides some degree of abstraction with regards to which tables/names, which columns/names etc. and just because a table exists in the DB it doesn't have to exist in your data context.

[)amien

I do kind of agree about this being an idealist approach with the separation, I am interested (in a somewhat morbid sense) to see how it works out..

It seems to me when you say - linq to sql will create its own classes, and that you already have yours defines.. however- why does your application care about the implementation of the object? why isn't your application model defines as interfaces? all your application model cares about is the signature of the data.. and it seems that the core of that data should well be from the data model, and not a representation of the data model..

@Damien:

>>>Wrapping those objects over LINQ to SQL like that is throwing away 99% of what it provides - update change tracking, relationship navigation, identity mapping, relationships between new objects, update sequence etc.<<<

Not sure I agree it's 99% - perhaps it's more like 98.5% :p. Seriously though, at this point in the game I don't want to deal with the DataContext, and I don't want "that much DB" in my model.

I might change my mind though :). We'll see.

>>>I get a feeling you're going to find life more difficult going forward as you reinvent some of this.<<<

In all fairness, I'm not reinventing anything - I'm just not using the generated classes. I am using Table<T> though, which for me is all I want.

>>>LINQ to SQL provides some degree of abstraction with regards to which tables/names, which columns/names etc. and just because a table exists in the DB it doesn't have to exist in your data context.<<<

I know I can massage, slightly, what Linq To Sql will do for me with respect to generation. But at some point I will have to compromise either my OO approach, or my relational approach to use it. I don't want to do that.

With simpler databases I agree - no need to abstract it. But my experience with eCommerce tells me that a store DB is anything but simple, and will grow heavily. There needs to be abstraction here.

@Stephen:

>>I do kind of agree about this being an idealist approach with the separation<<

Ha! Never been called that before :). But that's OK. It's true to a degree that I'm planning on the DB growing massively - but this is based on experience with the CSK.

>>>why does your application care about the implementation of the object?<<<

The model is part of the application, no? Maybe I didn't understand your question - yes I can throw an interface around - but i'm not sure how that's different then what I'm doing now.

Well, my point was you have two.. say... ProductDescription's, one that is the applications, one that is part of the ORM model..

You then map between the ORM one to the app version on your selects, so thats abstracted away nice enough.. but now the object your application uses isn't the database one.. and this could be important for relationships and instancing..

In terms of what your application cares about.. it wants a collection of products, and a product has a name etc etc.. it doesn't care that the products children are a Collection<T> or Table<T>, only that its enumerable (or queryable)..

It seems to me that the application should define interfaces that define a product and its relationships.. and then you would apply these interfaces to objects in the ORM (either directly on the generated classes, using partial classes) or on internal implementation classes that wrap around the ORM's classes..

As far as what your application see's.. it gets an IProduct.. it doesn't really care that under the covers.. the actual type is that defined from the ORM..

The only problem I see with this concept, is that for your app, the only workable models are all interfaces, and it has no way of making instances.. but this could be got around relatively easy, by asking the storage provider to create you a product instance (that it returns cast as IProduct)..

Sorry if that all sounds confusing.. I just think in terms of what the app wants.. it just wants an object that has x members with the signature it wants.. in terms of the plumbing of that object, that should perhaps be up to the underlying storage to say..

@Stephen: Thanks for the clarification :). I'm still not seeing the benefit here, to be honest. I want my model to be OO as much as I can - I don't see how interfaces fit that (but I sure am open to it).

By using an interface it seems you're allowing for multiple implementations of Product - I only want one, and that's the one in my model :).

>>> but now the object your application uses isn't the database one.. and this could be important for relationships and instancing<<<

That's precisely how I want it. My model doesn't need to know how it's stored - and the relationships in the DB have nothing (in theory) to OO relationships. At least that's what I'm after here.

I like this dialog - i hope I'm not missing your point here. You and I are on the same track I think...

Here's a dead basic overview of whats in my head about interfaces vs proxy classes for the app model:

Contracts Project

+ IProduct

- + Name

- + IEnumerable<IProductDescription>

Descriptions

+ IProductStorage

- + Products

- + CreateProduct

- + DeleteProduct

- + StoreProduct

Database Repository Project

- References Contracts

- Using sql metal generated classes

- + Explicitely implements IProduct from the contracts asm and simply returns the Name and Descriptions property that is defined on the orm object already

- + Explicitely implements IProductStorage on the DataContext, and maps the required functions to the equiv of the data context (optionally, instead of implementing on the datacontext, is instead a new class that proxys the datacontext)

Main project

- References the contracts project

+ ProductManager

- + Takes an IProductStorage at the ctor

- + Optionally contains a static Current property that returns a manager with a default IProductStorage

The IProductStorage then has basic functions of being able to give you the unfiltered data, the manager never makes reference to the ORM classes (they can be internal since the IProductStorage returns IProduct)..

This way, the main project is working with IProduct, which actual instance is that given from the ORM, the main project will never know about this..

Thanks again Stephen - can you tell me what I gain by using interfaces versus a static model, aside from things not knowing about other things. Specifically, I've abstracted the mapping already, tell me what adding another abstraction allows me to do.

Also - using this pattern it's possible to diversify the model, so I could implement IProduct anywhere I need to. Is this intended as well?

Rob ? could you reply to my post ?

how do you see the update/delete/insert functions ?

and how will you deal with relations ? my feeling is that you will not use linq entity capacities.

So, the challenge for me would be to keep performances and obtain good design (DDD or service layer or ?) by using linq 2 sql (for the foundation) and specifically linq 2 sql entities.

But maybe it just not possible and we do use nhibernate (with nhibernate.linq) or subsonic (with linq to subsonic... one day)

many thanks to you

Sorry, I'm terrible at explaining..

Currently on your current source code release, you have two category classes, one that is the representation of a category for the storage model, and one that is the category for the application model..

Commerce.MVC.SqlRepository.Data.Category

and

Commerce.MVC.Data.Category

The repository for the sql spike, is the one responsible for getting a storage model category, and creating an application model category..

---

return from c in ctx.Categories

select new Category

{

ID=c.CategoryID,

Name=c.CategoryName,

ParentID=c.ParentID ?? 0

};

---

Seen there.. the reason for this, is you want your application model to use the category class you defined.

At this point, at this point, the category object from the data model is left behind (or essentially not really created).. however, that class contains notifications to the datacontext via propertychanging events etc.. these aren't of course required, but they do aid how clever the datacontext can be.

My idea was that the category class (for example) in the application model, is changed to an interface, and thats all the application would care about, the sql metal generated category would implement that interface, and the repository would simply be returning the category object that the orm provides, but as far as the application model see's.. its got its ICategory.. when it does changes to that category, its actually changing the data models category and so the link isn't lost.. the link of things like the datacontext keeping hold of single references of 'rows' (so you could pull the same row out twice and compare them to be the same, vs yours that wouldn't be the same reference).

More than this being anal, I was just wondered why you want to detach the application model away so much.. surely the important part is ensuring the application model doesn't tie itself to the storage model.. which interfaces will ensure..

(maybe I'm talking rubbish!)

@sergei:

>>>how do you see the update/delete/insert functions ?<<<

I don't see a problem here - i do the reverse, in a sense, of what I'm doing now.

>>>and how will you deal with relations ? my feeling is that you will not use linq entity capacities<<<

I can run joins just fine under the hood - the example you're seeing is me just hitting one table; with Products I'll probably hit 5, joining them and returning what I need.

>>>So, the challenge for me would be to keep performances and obtain good design<<<

Well, if I can channel Ayende - perf is a feature. I can work that in later and don't need to worry about it now. That's not really me, but at the same time it doesn't equate to Linq To Sql. If I'm really worried about it, I'll work on caching, not shaving the milliseconds from another call to the DB.

>>>But maybe it just not possible and we do use nhibernate (with nhibernate.linq) or subsonic (with linq to subsonic... one day)<<<

Maybe ... :)

Rob this cast just felt tons better (I don't know why)

Passing around IQueryable is great (but feels funny) and you are locked down to "LinqTo..." how would you implement your class with any other ORM tool say SubSonic or will this require that anyone that wants to create a provider for MVC.SF use IQueryable<>.

@Rob

What i say is :

if you create your model objects, you have to track your change (unit of work)

so in the case of customer, if i want to change some proprieties, in your solution, i have to load Customer object, change its proprieties (and no tracking of changes ?) and give it to service layer

then, the service layer initiate DB, load customer entity and what ? you copy the values from model customet to linq entity customer before write modifications in database ? without implement a unit of work for your model objects (to keep only changements => change tracking)

same note about the lazy loading, for example : customer have many orders or category have many products

with linq 2 sql, it's out of the box

in your solution you have to reimplemente it

it the reason why i think that you use only one part of linq 2 sql, it's a pity for me.

and so, for me, the challenge is to set up an architecture base on linq 2 sql with a plenty of use of its functionnalities.

Maybe it's not possible or not completely and we have to use a real ORM that deal with your model objects.

@Stephen:

>>>however, that class contains notifications to the datacontext via propertychanging events etc.. these aren't of course required, but they do aid how clever the datacontext can be.<<<

True, but I don't want the DataContext to be part of my application, and I don't want change-tracking, not yet anyway.

>>>I was just wondered why you want to detach the application model away so much.. surely the important part is ensuring the application model doesn't tie itself to the storage model.. which interfaces will ensure..<<<

What I want is a representational model, based on OO, separate from the DB. I don't see what interfaces buys me here.

@sergie:

>>>if you create your model objects, you have to track your change (unit of work)<<<

Unit of Work aside, I have no case for this yet. If I do, I can implement INotifyProperty as needed. I don't want to rely on LinqToSql to do this for me. If it needs to be part of the model, it needs to work for every repository right?

Not sure why you think I can't use the Lazy loading and other things from LinqToSql - I certainly can, and will :). Just not the way you've seen :)

I don’t fully agree with your implementation, I’m sorry; I’m a big user of Domain Driven Design and the Repository Pattern. In your case you return a Queryable object not the domain object an isolated domainobject. The Repository itself shall give you methods based on your Where clauses. Like GetCategoryByID, GetCategoryByTotalOf etc… etc…

The Application/Service layer on the other hand is more like a façade between the UI layer and business logic and the domain (Domain layer within DDD). In your case you put the business logic inside your service layer the logic that should live within the repository. The Repository also supports the objective of achieving a clean separation and one-way dependency between the domain and data mapping layers.

What you are doing is an object (your repository) that returns a “ready to query” domain object (entity). Later on as it seems you use a filter object to do your Where. The “where” are the Repositories methods in DDD. Another thing is also that an Repository shall isolate the data mapping for you, when an domain object return from the repository it shall be data mapped, in your case you call the data mapping inside your service layer which is kind of wrong. I think you got the picture all wrong, I’m sorry. Your code doesn’t follow some of the main principles regarding OOP when doing this.

The separation of concern regarding data persistence and service layer are also broken in your solution. It’s not that simple to mock or remove the repositories and replace them with other Data Access models when you got that hard coupling with Linq.

@"Moses" :)

>>>I’m sorry; I’m a big user of Domain Driven Design and the Repository Pattern<<<

No need to apologize for that, you're among friends here :)

>>> The Repository itself shall give you methods based on your Where clauses. Like GetCategoryByID, GetCategoryByTotalOf <<<

If I did that, I'd have a massive Repository, with a massive interface yes? One method for every possible "GetThingByX"? Is this really a scalable design wherein you can create more Repositories as needed?

I'm putting all the logic in you "GetBy" methods into one spot - how is that not a good thing?

>>>In your case you put the business logic inside your service layer the logic that should live within the repository<<<

The Service Layer exists for Business Logic - I don't see the benefit to putting anything like that in the Repository.

>>>What you are doing is an object (your repository) that returns a “ready to query” domain object (entity).<<<

Not really - I'm returning an interface to the Repository. The lightest, cleanest interface possible.

>>>The “where” are the Repositories methods in DDD<<<

Agreed - but we didn't have IQueryable before, and I'm going to bet DDD changes to start using it.

>>>Another thing is also that an Repository shall isolate the data mapping for you, when an domain object return from the repository it shall be data mapped, in your case you call the data mapping inside your service layer <<<

No - the mapping is done in the Repository (it might look confusing in the webcast - have another look) - in the SqlCatalogRepository to be specific.

>>>The separation of concern regarding data persistence and service layer are also broken in your solution. It’s not that simple to mock or remove the repositories and replace them with other Data Access models when you got that hard coupling with Linq<<<

My use of Linq doesn't "break" the concept. I agree that IQueryable isn't part of the landscape very much these days, but give it 6 months or so - LinqToNhibernate will be done, and so will SubSonic 3.0, both of which will implement IQueryable.

Have more comments, but will comment only some for now.

>>The Service Layer exists for Business Logic - I don't see the benefit to putting anything like that in the Repository.

Application/Service layer (to names for the same layer)

Defines jobs the software is supposed to do and directs the expressive domain objects to works out problems. It does not contain business rules or knowledge, but only coordinates tanks and delegates works to collaborations of domain objects in the next layer down. It does not have state reflecting the business situation.

>If I did that, I'd have a massive Repository, with a >massive interface yes? One method for every >possible "GetThingByX"? Is this really a scalable design >wherein you can create more Repositories as needed?

No. It all depends. Do you really need many GetThingsByX in an OOP software, I say no. I have build some large systems and used the Repository Pattern as it is defined and don't got that much methods in them I normally got.

GetThingsByID, GetAllThings, SaveThing, UpdateThing, DeleteThing then in some cases some more methods. The thing is if you do OO and a nice OOP and a clean model after DDD you then use the objects to do stuff. All Repositories are created after their Aggregate root entity objects.

And often you want to have your objects as POCOs )Plain old CLR objecs) and as with the IQery... you don't have em POCO. And what about applications when you don't use ORM?

>Agreed - but we didn't have IQueryable before, and I'm >going to bet DDD changes to start using it.

How hard is it for DDD ppl to build their own IQueryable look alike solution if they wanted it?

>My use of Linq doesn't "break" the concept. I agree that >IQueryable isn't part of the landscape very much these >days, but give it 6 months or so - LinqToNhibernate will >be done, and so will SubSonic 3.0, both of which will >implement IQueryable.

True, but what does that have to do with the Repository pattern? You capsulate the data access logic within the repositories so your domain, application/service layer and UI layer don't even need or want to know how you got your objects as with DDD.

Your solution is nice, but I think it’s really sad you use a word and a named pattern that already exist and then implement it as it should not be implemented.

//Moses

One more comment before bed time :)

>What I want is a representational model, based on OO, >separate from the DB. I don't see what interfaces buys me >here.

What is OO for you? As I see it you don't got any Associations whithin your domain object, you use public "int CategoryID { get; set; }" instead of:

"public Category Category { get; set; }" as within OO(P).

OO is not a table driven design. Tables within RDB aren’t OO classes. You will got mayor design problems within the OO world I promise. OO are mostly a reflection of the real world objects and there aggregation and association with each other.

As with your "public int CategoryID { get; set; }" You must do this in relational databases though they can't use associations which is a part of the OO design. This is a major problem many .Net developers do all the time...

what’s your thoughts about that?

@Moses:

I'm not sure what you're referring to RE OO and the DB - perhaps I didn't make it clear enough that my Model abstraction is all about keeping OO where it belongs, and relational theory where it belongs (in the DB).

WRT to your Category suggestion - I don't have any tests that tell me to have a Category as part of my Product - yet. That will be coming :).

I haven't claimed this to be a DDD application, and I appreciate you taking the time to offer me some thoughts thus.

As far as the way I've implmented my Repository - I don't see how it's "sad" really. It's very testable, and doesn't break when I change another part of the application (my DB). Testable and highly encapsulated - this is the goal I'm shooting for.

@Rob

about performances...

"Well, if I can channel Ayende - perf is a feature. I can work that in later and don't need to worry about it now. "

No, definitely no, this assertion is wrong !

Optimization is a feature, and not performance

To reach performance, nowadys, you have to reach scalability and the grail is linear scalability

So performance is a foundation not a feature

I have worked on a website with more than 100000 simultaneous users

These users can chat (messenger), make event, make search, send mail, etc on huge database (50M of members) and i can tell you that performance wab though at the beginning of the project

Can you think that google, facebook, myspace(, my website) think of performance as a feature ?

I repeat again, nowadays, performance of a website is the capacity of this website to be scalar and this is a design decision so you have to considere this when you suggest an architecture.

I think linq change a lot of think, because of iqueryable interface, projection, etc and i hope it to get us to a new architecture or a major evolution in order to reach more pragmatism when design architecture.

Thanks Sergei :). I understand scaling quite well (I made two sites for PayPal). 100,000 simultaneous is quite the site! I don't think Facebook + MySpace + eBay combined could say that...

All that aside - you're assuming that I'm ignoring performance, and I'm not. I just haven't gotten there yet.

>I haven't claimed this to be a DDD application, and I >appreciate you taking the time to offer me some thoughts >thus.

That's true, but you refer it as a Repository Pattern as within DDD and as Fowler also refers too.

And if you are implementing after their description then you do it wrong :( if you merge your service code within the repository instead then you got it.

>I don't see how it's "sad" really. It's very testable

Everytihing is very testable :)

The biggest problem is often mocking in testable.

And in your case you have some problem testing SqlCategoryRepository though you got the hard coupling with the Datacontext within it. So you can’t unittest that class without accessing the DB.

public IQueryable<Category> GetCategories() {

DB ctx = new DB();

so try to set the datacontext within the constructor of your repositories so you can mock it if you want too. if you pass the Datacontext within the repository you aslo got the benefit to reuse the same datacontext within different request. Eg. Say you want to use the UoW thing within a wizzard scenario on a webpage. If you put datacontext in a session and send it to the repository you got fully UoF within the wizzard steps. This you can't do right now.

> I don't see how it's "sad" really.

It's ok coding and how to code it individual and up to you, But I think you shall call it something else then a repository pattern when this name is taken and the use is really clear how to use it. That's the sad part, you are breaking the description and the isolation about the well known Repository Pattern.

You have made a query pattern of some kind and call it Repository. You have moved some repositories main responsibilities to a higher layer.

You moved some of the Repositories responsibility and put it in your service layer (where it shouldn't exist). Why not put that service code within your repository instead? You will get a better abstraction and separation for each layers responsibility if do so. Now your service layer owns the knowledge how to return a Category to your User interface, your service layer acts exactly as a repository should. So if I was you I should have renamed the CatalogService layer and call it CatalogRepository. Then add another CatalogService class that calls this CatalogRepository instead. SqlCatalogRepository can be renamed to SqlCatalogDataMapper as it seems to be an implementation near the Data mapping pattern.

You layers will be.

UI ---> CatalogService (your new class) ---> CatalogRepository (former CatalogService) ---> SqlCatalogDataMapper (former SqlCatalogRepository) (pluggable) -- > Datacontext

Eg. Of CatalogSerivce

public CatalogService(…)

{

public IList<Catrgory> GetAllCategories()

{

try{

ICatalogRepository catalogRepository = new RepositoryFactory<ICatalogRepository>();

return(catalogRepository.GetCategories());

}

catch(<*>Exception ex)

{

//log if needed, throw ApplicationException…

}

}

}

As you can see the code got less lines of code, it’s more readable, you can add the logging things within your service layer where it’s often shall exist.

I use a RepositoryFactory (use eg. Spring .Net(object builder) then you can let the DI/IoC framework decide what Repository it shall use and what Datamapping object, eg. SqlCatalogDatamapping, or OracleDatamapping, or NHibDatamapping etc… The Datamapping shall got an Datacontext so you can mock that.

public CatalogRepository(ICatalogDatamapping datamapping)

{

ICatalogDatamapping _datamapping = datamapping;

public IList<Category> GetCategories()

{

>All that aside - you're assuming that I'm ignoring >performance, and I'm not. I just haven't gotten there yet.

Damn I really start to like you better and better :)

Nice. I agree with you, it's waste think performance before you are done and really can test your code in a real scenario. Ppl lay about 70-80% of time wasting on performance tuning when it's often not needed. So code YAGNI and KISS first, when done, do performance test then tune if needed... Nice approach saves me many lines of code, time, and money and deadlines :)

Loved your answer there...

Sorry for my terror, but saw that my long message with code example got cut so here is the rest of it :-/

public CatalogRepository(ICatalogDatamapping datamapping)

{

ICatalogDatamapping _datamapping = datamapping;

public IList<Category> GetCategories()

{

IList<Category> rawCategories = _datamapping.GetCategories().ToList();

var parents = (from c in rawCategories

where c.ParentID == 0

select c).ToList();

parents.ForEach(p =>

{

p.SubCategories = (from subs in rawCategories

where subs.ParentID == p.ID

select subs).ToList();

});

return parents;

}

}

Something likes that. But remember you call the repository Catalog* but get Categories? A repository shall always return the same domain/model/entity as its notation. So in this case the right name should have been CategoryRepository not CatalogRepository.

As you can see, very pluggable code, mockfriendly, more testable then your code thank’s to the mocking of Datamapping/datacontext etc… Less lines of code, better single responsibility and separation of concerns between the layers and their responsibilities.

@Moses: I'm having this same conversation with 3 people, and I think it's fair to say I'm working on a NEW Repository pattern - let's call it Repository++ :):).

@Moses and rob

I think there is a confusion between performances and optimization

It's obvious that architecture is the base of performance, the base.

I've noticed and experimented many times that great design and great scalabilty have to be thought at the beginning

It's not incompatible with a nice design, it's why i don't talk about optimization

By example, linear scalability coult not be obtain easyly with cache component

Also, cache must be WELL though otherwise you risk inconsistent inside cluster and side-effects.

About the 100k simultaneous users, i hope that myspace, facebook and others have more, much more than 100K users :)

But i fully agree with you with optimization is a feature.

I add that a linear scalability is the foundation of a well great architecture design because it would say that your architecture is based on components which are autonomous and could be independently scalable and independently grow, regardless of the technology, the core, etc

My ambition is to reach this holy grail :) with DDD, a DDD that could be change with linq new possibilities.

So i'm very attentive to the architecture that you suggest but i think my little experience on scalable architecture can make sense...

>@Moses: I'm having this same conversation with 3 people, >and I think it's fair to say I'm working on a NEW >Repository pattern - let's call it Repository++ :):).

He he ok... :D Then why not create your own name then? Skip the name repository pattern cause it's not sorry but it really isn't even if you think so... Trust me... I also think the 3 others are on my side here too and are fan of Fowlers pattern and the Repository pattern that came from DDD.

But you also kill the service layer definition ;-) and it's representation when adding all that business rules there. You are a really persistent person :D

But I also think you are rather new in this topic and patterns too. But it’s ok we must always start somewhere :D …And I hope MS never release that as a Pattern and tell people that it’s how a repository pattern shall be build because it’s really wrong in many ways. Good luck!

sergei:

I have build many application with DDD and no performance in thougts, thanks to nice separation, good code design, refactoring, principles etc... It's very easy even too easy to add stuff that increase performance if needed and also throughput etc...

But that's another topic... :D

this is a really great series Rob. I've just finished watching the first four, and have been updating a simple ASPNET.MVC blog application to help me learn as I go along. I had decided also to try using VistaDB to get a feel for its capabilites. Unfortunately, things fell apart when I needed to return an IQueryable<> interface. Is there a nice and simple way I can do this without having to implement IQueryable myself? I realise that for the moment I will have to make do with it always getting the full list from the database and letting linq to objects do the filtering.

@Mark - you can try LinqToNHibernate, not sure if it's ready yet but that might work for you. Sorry to say that at this time IQueryable isn't implemented everywhere yet.

I'm working on this for SubSonic 3.0, as a response to this series :).

thanks Rob. I'm really looking forward to seeing what's in SubSonic 3.0. After changing all my IQueryable's to IEnumerable to get it compiling, I suddenly realised I could simply use the AsQueryable extension method on a List. OK performance will probably be horrible, but its just a test app so I'm not too bothered.

@Stephen – I’m not sure if I’m following you correctly, but it seems to me that making the business model into interfaces might be counter productive in the long run. For example, let’s say we have an ICart which has a TotalPrice property. TotalPrice would not be a value persisted to any data store, but would be a calculated value (i.e. sum of all items in cart * tax and delivery charges – discounts). It seems to me that by making ICart into an interface you would in effect be moving the actual business logic out of the domain layer and into the data access layer. This also means that the actual logic for calculating TotalPrice would need to duplicated for any future repositories. What do you think?

@Rob – First, there’s some really great stuff here. There’s a lot about your approach that I really like and a fair amount that I disagree with, but it’s really nice to see another developers thought process when building an application. I think as developers we tend to think in Booleans. This way is right and that way is wrong. The truth is, every design decision is a trade off (maintainability vs. performance or scalability vs. cost of development). We’re all trying to find the right balance for our projects.

That being said, I have a question about your decision to not make your domain model match you’re data repository. For me, I’ve always really liked the theory that the actual data schema doesn’t have to match the domain model but in practice have rarely found this to be practical. Specifically, I’m wondering about not having a domain object that mirrors the localized product description table. It seems to me that at some point you will need to build an admin interface where users can enter/edit the product description in each language supported by the application. Won’t you need to create a domain object for the product description table at that point? What happens to the existing Product object at that point? Are you thinking there may be separate domain objects for an admin application? I’ve often about that approach, but it seems to me that this would go against DRY principals.

What about abstracting the idea away at the domain level as opposed to the service layer? Specifically, what about having a read-only LocalizedDescription property on the product class that handles getting the correct ProductDescription object and returning it’s Description property? Could this save future repositories from having to recreate logic for returning the correctly localized product description? It could also have benefits when it comes time to add caching. In the current implementation, won’t you need to cache a separate instance of a product for each language supported? Is there an overriding design objective that I’m missing? Am I misreading the code?

Again, this series is outstanding and I’m really looking forward to seeing where it goes.

Yes I agree there are inherant issues with pushing all the expected logic into interfaces, the idea for me was that the data entity would only represent the basic data. Any composition of that data would be contained still within the applications logic, granted this would make it somewhat less clean, as instead of asking the cart for its total, I would be asking say.. the cart manager, whats the total of this cart..

This can be cleaned up slightly with extension methods.

The main issues I saw with it myself are that your app model has no class it can create.. it cannot create a product, it has to instead ask the data model to make a product for it..

For my experiment here, I found that interfacing the business model up into the app model, and ensuing those interfaces only specified storage data, no composition of data.. and then building extension methods around those interfaces to define composition logic. It saved a lot of potential nightmares that I could see coming with mapping everything across to the app model..

Relations and collections were the biggest issue.. how do you handle the differences between getting a product from the database, adding some description children to it, then saving that product back.. compared to creating a new product, adding some description children, and adding that back.. it just seems like your are either heading into a nightmare.. or you take a really easy route out by cutting out lazy loading, and prepopulating a ton of data, and then posting it back to the data model to get evaluated..

Then comes into the question about handling concurrency..

I'm really interested to see if the solution that comes out of this is clean.. but personally I thought my experiment cut down on a lot of headache.

Also, I should say that abstract classes would act as an alternative.. these can be used as proxys in the event you cannot control the inheritance of your business model..

I have to say I'm with Moses in that constraining yourself to a IQueryable Repository is not really applying the Repository pattern. Anyway, I DID like how you abstracted your model from LinqToSql objects. I hope it doesn't bite you back. :)

"WRT to your Category suggestion - I don't have any tests that tell me to have a Category as part of my Product - yet. That will be coming :)."

They told you when you decided to add a CategoryId. This is something your Product shouldn't know about.

Oh, almost forgot.

"an integration test is pretty much shorthand for "hitting the database"

Well, it's not exactly that. It's a test that test two or more systems interacting together. It can be your system under test interacting with the db or for example, with another object of your domain.

Hi Rob, first off, thanks for great series of posts! A couple of quick questions:

Have you considered the alternative of using a mapping file to map directly from your model classes to the db, rather than having both the model classes and the sqlmetal-generated classes?

Is there any particular reason why you're not having linqtosql manage the relationships between the objects?

This might be a bit too early, but since you're implementing the full DDD repository pattern, how do you intend to handle persistence of objects within aggregates that are other than the aggregate root? For instance persisting OrderItems and ItemParts in an OrdersRepository...?

Hi Jimit:

>>>Have you considered the alternative of using a mapping file to map directly from your model classes to the db, rather than having both the model classes and the sqlmetal-generated classes?<<<

If you mean using an alternate ORM tool - yes. At this point I want to keep it simple, and all I'm looking at are about 10 extra lines of code to handle this operation so I'm not worried yet.

>>>Is there any particular reason why you're not having linqtosql manage the relationships between the objects<<<

Yes, I go into this is part 4, 5, and 6. I don't want DB functionality in the model, and I want to keep it as OO as I can. RE persistence of aggregates - i'll deal with that when the time comes :)

Rob,

can you create one blog post about Repository pattern to dissect it in a more detail, and with some guidelines (not related with this blog series)

Nice green lawn to take a nap on :)

An alternative to returning IQueryable is accepting Expressions in the repositoriess Find() methods. That way, you assure that the database will do most of the filtering and return the least amount of data needed.

I feel that IQueryable is still associated with the actual ORM used and so it should not be passed up from the data layers.

I'm learning too, so how do you feel about my comment?

Thanks!

I am a student from chinese ,I am english is poor,please speak slowly in your video