Sql server functions

Sql server functions

Old forum URL: forums.lhotka.net/forums/t/1567.aspx


ajj3085 posted on Monday, October 23, 2006

Hi,

I have a quote info object, which just displays summary information about the quote.  The details (like the line items) aren't loaded and not needed, but I still need to get data like Subtotal and Tax amounts. 

So my question is this; should I just create Sql Server functions to compute totals and return the result as a column in a view, or should should object build the query in the dataportal fetch method?  I'm leaning toward the former, since this will give better peformance, but it also feels like some business logic is going into the db.

Thanks
Andy

Bayu replied on Monday, October 23, 2006

Why not pre-computing these values and store them alongside your quote details?

This would imply that you have to compute these every time you alter your quote (add/remove line items, change discount rate, etc.) but that should not be a problem. The pro of this approach would be that you alleviate both your BO and DB from a lot of computational processing everytime a list of quotes is requested.

In case of an internationalized app (i.e. with different currencies) this would still work fine with some conversion table. You store a base-price in dollars (euros) with your quote details and using a multiplication against your conversion table in your sql you can return the the value in the desired currency. You might wonder: what benefit remains? Well, from a DBMS perspective, aggregate values like computed summaries are MUCH more complex (performance intensive) than a single straightforward multiplication per row.

Bayu

ajj3085 replied on Monday, October 23, 2006

Thats another idea, although I always worry that the summary column will get out of sync from the actual values.  Globalization is not a concern for now.  I'll investigate that approach.

Bayu replied on Monday, October 23, 2006

:-)

I understand your concern. It's also against normalization rules actually, storing the total price with the details is of course redundant information and therefore considered 'asking for trouble'.

If you can be sure your BOs are the sole party ever to change Quote details it should work fine.

Btw: another option is to implement a trigger. Personally I never used this, so I have no idea how to implement it. But you should be able to implement some procedure/function in your DB that is triggered every time you insert/update/delete Quotes and/or line items. This function could then maintain a simple two-column table of computed Quote-totals (on column with a foreign key to Quote, and one with your amount).

Considering this: the advantages would be mulitple:
- you have the high performance of your DB
- you only update totals when needed, which is actually what I was after in my previous post
- you are robust to (future) other applications that may modify your Quote data as the logic is in your DB

I can't help you further on this 'trigger' stuff though, for I have zero experience with those.

Regards,
Bayu

ajj3085 replied on Monday, October 23, 2006

I've thought about the summary columns, and I don't think they'll work.  One of the things we are trying to do is keep pricing accurate.  If a quote is created and saved, then later a product has its price changed, it should show when the quote information object is loaded.  So its a good idea, but I don't think it will work here.

Thanks
Andy

Bayu replied on Monday, October 23, 2006

Right, then you indeed just create trouble.

Then I would opt for using standard aggregate functions (sum/avg/min/max/count) and trust my DBMS to opmize its query evaluation plans appropriately.

Bayu

ChristianPena replied on Monday, October 23, 2006

Andy,

I would put the calculation in the view. Besides the stated performance benefit, it allows you to make changes to your calculations without having to make changes to the object.

The issue with storing aggregate columns is that you have to update your schema in order to stay current with requirements. If your users wanted to see a further breakdown of your tax amounts (arbitrary example, not sure if it applies), then you would have to store a new column for each and update each accordingly. If you modify your calculations then you have to update the affected aggregate columns.

Christian

ChristianPena replied on Monday, October 23, 2006

You guys type quickly. :)

ajj3085 replied on Monday, October 23, 2006

Christain,

Thanks for your comments.  I think I'll go the view route.  as you point out, additional changes could create additional columns.

Bayu, triggers can be messy, even when they do what you want.  There's also the case that at a certain point the document is locked, and prices in affect at that time should now NOT update when changes are made the product prices.  So the trigger would have to figure out which documents aren't locked and update the summaries there as well.. I think that would be a bad thing.

I appreciate the comments though, it helps to have others to bounce ideas around with.

Bayu replied on Monday, October 23, 2006

Yeah,

I have been googling a bit on 'triggers' since I wanted to know what I was actually talking about. ;-)
They are messy indeed, so yes: ignore that comment. :-)

I agree with you that the view-approach seems to be the best approach. As Christian pointed out this will have two benefits:
- changes in computational logic are hidden from the BO and UI
- you deal adequately with changing product prices.

In addition:
- your DB schema remains neatly normalized

I wasn't too comfortable either with storing redundant info. But like you say: this forum is great for test-launching ideas. ;-)

Grtz.
Bayu



rbellow replied on Monday, October 23, 2006

Personally I would create a line item collection under your Quote info object. It would eliminate any normalization issues and encapsulate all your business logic within your object. I have used the summarization approach at the summary level before. The main problem is that unless your documentation and processes are solid, other developers may corrupt your data. Then you will have to build additional processes to validate that data. It becomes a nasty cycle.

ajj3085 replied on Tuesday, October 24, 2006

Well I was hoping not to load the line items, because I don't need to display them, just create summary information for them. 

I have no need to worry about other developers... just me here, which is why I come to this forum so much. Smile [:)]

david.wendelken replied on Tuesday, October 24, 2006

Triggers are a perfectly reasonable solution to maintaining synchronization between source data and it's denormalized (and optionally aggregated) copy.

They have the virture of putting all the behavior for synchronizing the data in one place instead of in (possibly) many objects that may each serve just one use case. 

Frankly, I believe taht manipulating business data is the business object's job.  Manipulating copies of data in the database made for performance reasons is the database's job, it's not a business activity at all.

 

===== second, unrelated point =============

One person made a point on this topic, and I strongly disagree.

From a business point of view, I would not think that changing a quoted price for a product on a quotation line item when the product's price changes is a good idea.  If you do that, then you will have some very unhappy customers!  You will have quoted one price to them and then, when you look it up later, you will think you quoted them a different price.  Bad, bad, bad!  A guaranteed way to make your potential customers angry if the prices have risen since you issued your quote - they will think you were lying to them.  And a great way to lose money they were willing to pay if prices had fallen since the quotation was given.  A quotation expiration date is the usual way to deal with that - the prices quoted are good until the quote expires.

 

ajj3085 replied on Tuesday, October 24, 2006

david.wendelken:
From a business point of view, I would not think that changing a quoted price for a product on a quotation line item when the product's price changes is a good idea.  If you do that, then you will have some very unhappy customers!  You will have quoted one price to them and then, when you look it up later, you will think you quoted them a different price.  Bad, bad, bad!  A guaranteed way to make your potential customers angry if the prices have risen since you issued your quote - they will think you were lying to them.  And a great way to lose money they were willing to pay if prices had fallen since the quotation was given.  A quotation expiration date is the usual way to deal with that - the prices quoted are good until the quote expires.


David, that person was me, and this is the requirement, although it sounds like you missed part of the requirement.  Prices are updated up until the document is locked.  The document is only locked when the quote is sent to the customer.  After that point, the price remains fixed at whatever it was when the quote was locked.  (Indeed, once a quote is locked, it can never be changed again.  Changes require a revision, which is a new document with the same number + a revision number).

A quoted price is only good if it was sent in writing to a customer; prices given over the phone are not guaranteed.

Now given this requirement, it feels like the trigger is not a good place to update documents, since the database now has to know the different between a locked and unlocked document.

Hope that clears up my requirements.

Andy

david.wendelken replied on Tuesday, October 24, 2006

You are right, I did miss part of the requirement.  I missed the distinction between "quote-in-preparation" and "quote-sent-to-customer".   :)

That said, you have one or two places for keeping prices up-to-date.

1) When a product updates its price, you need to copy the new price to un-sent quotations.

This is NOT a denormalization issue.  The quotation line item price and the product current price are two different pieces of data.    Thus, this is a pure business logic issue and the code should be invoked from the business object.  Naturally, the business object needs to distinguish between sent and un-sent quotations.

Alternatively, you could modify your database design and store sent quotations in one set of data tables and quotations in-process in another.  When a quotation is sent, it copies the current product prices and moves itself to the sent quotation tables (and removes itself from the unsent ones.)  The in-process quotation line item would not actually have a line item price, as it would look up the product info for the base line item price.  This would avoid the entire update the unsent quotation line item issue entirely.  It also has the advantage of avoiding fragmentation of your quotation table storage area.  Sent quotations shouldn't (normally) be changed, whereas unsent quotations might be changed a lot (depending upon your business) as you dicker with the customer over requirements. 

2) If you choose to denormalize the totals into the quotation table from the quotation line item table:

Every time you update a quotation line item price, the denormalized total needs to be updated.  For this purpose, sent/unsent is of no consequence.  (If you allow a change to quotation line item then you have to update the totals, period.)  This is a pure database issue and a trigger would be excellent here.   No matter how many different objects mess with quotation line item data, the database can keep them synchronized.

Optionally, there is an option in Sql Server 2005 (maybe earlier) to define a computed column.  Haven't double-checked how it works yet, but it might do the trick for you.

Bayu replied on Wednesday, October 25, 2006

david.wendelken:

Alternatively, you could modify your database design and store sent quotations in one set of data tables and quotations in-process in another.  When a quotation is sent, it copies the current product prices and moves itself to the sent quotation tables (and removes itself from the unsent ones.)  The in-process quotation line item would not actually have a line item price, as it would look up the product info for the base line item price.  This would avoid the entire update the unsent quotation line item issue entirely.  It also has the advantage of avoiding fragmentation of your quotation table storage area.  Sent quotations shouldn't (normally) be changed, whereas unsent quotations might be changed a lot (depending upon your business) as you dicker with the customer over requirements. 



I like this one very much.

It mirrors the fact that 'sent quotations' are hugely different from 'in-process quotations'. By keeping them separate you clear up a lot confusion, as probably a lot of your relations to quotations (maybe even all of them) are only relevant for (exclusively) one of the two types.

Additionally, you can easily apply different security grants to each type (i.e. table) and also this separation will naturally lead to different BOs which each serve their own use cases.

[edit]

Thinking about this some more: you could still support relations that are relevant to both types of quotations by having 3 tables.
- Quotations {QuotationID, Status)  // status can be 'in-process' or 'sent'
- SentQuotations(SentQuotationID, QuotationID, ...<details> ...)
- InProcessQuotations(InProcessQuotationID, QuotationID, ...<details> ...)

I use this a lot for Person data (one table with ALL persons, and separate tables for specific kinds of persons, like Employees, CustomerContacts, etc). If you want a list of all quotations with all details, use OUTER JOINS to merge the tables.


Bayu

ajj3085 replied on Wednesday, October 25, 2006

That would bring me up to nine tables, all with the same data structure..  not very fond of that idea.

Bayu:
I use this a lot for Person data (one table with ALL persons, and separate tables for specific kinds of persons, like Employees, CustomerContacts, etc). If you want a list of all quotations with all details, use OUTER JOINS to merge the tables.


I do this as well, but the employee table only contains the additional data elements that are needed for an employee, it doesn't contain the same data as the person table.

I try to take great care not to violate normalization rules, although I realize there are times when it is helpful.

ajj3085 replied on Wednesday, October 25, 2006

david.wendelken:
1) When a product updates its price, you need to copy the new price to un-sent quotations.

This is NOT a denormalization issue.  The quotation line item price and the product current price are two different pieces of data.    Thus, this is a pure business logic issue and the code should be invoked from the business object.  Naturally, the business object needs to distinguish between sent and un-sent quotations.

That's how it is in the database as well.  Currently I'm copying the price into the line item table whenever the quote is saved; maybe I should only copy the price information to the line item table when the item is locked.  The BO's could then use the line item price only if its there, otherwise use the product price.  (Maybe I need a list of just product ids + their prices for a price list..)

david.wendelken:
Alternatively, you could modify your database design and store sent quotations in one set of data tables and quotations in-process in another.  When a quotation is sent, it copies the current product prices and moves itself to the sent quotation tables (and removes itself from the unsent ones.)  The in-process quotation line item would not actually have a line item price, as it would look up the product info for the base line item price.  This would avoid the entire update the unsent quotation line item issue entirely.  It also has the advantage of avoiding fragmentation of your quotation table storage area.  Sent quotations shouldn't (normally) be changed, whereas unsent quotations might be changed a lot (depending upon your business) as you dicker with the customer over requirements.


Well, I'm not a fan of denormalization to begin with.  Keep in mind that I also have Orders and Invoices, whose line item elements function identically to the Quote.  I'll also very likely have an RMA document in the next phase, although i'm not clear if it will be function similar to the existing documents.  So with this method, i'm up to six identical tables.

david.wendelken:
Every time you update a quotation line item price, the denormalized total needs to be updated.  For this purpose, sent/unsent is of no consequence.  (If you allow a change to quotation line item then you have to update the totals, period.)  This is a pure database issue and a trigger would be excellent here.   No matter how many different objects mess with quotation line item data, the database can keep them synchronized.


That would be a fine solution as well, except that product price changes won't be reflected, which was a problem when this solution was originally suggested.  Its not so much of a problem if the user's have to open the quote for editing to see any data, but that would be a poor user experience I would think.  If they have a list of quotes (which are read only objects) it would be ideal to give them summary data there (which is actually why I posted this; the editable business objects load everything and thus always get the correct data.. so its only an issue for my readonly summary objects).

david.wendelken:
Optionally, there is an option in Sql Server 2005 (maybe earlier) to define a computed column.  Haven't double-checked how it works yet, but it might do the trick for you.


Ahh!  At my last employer, they used Filemaker for their DB of record.  Calculated fields were one of the most problematic elements of that system I had to deal with (behind the lack of support for transactions, lack of referental integratety support a 'table' value columns).

david.wendelken replied on Wednesday, October 25, 2006

ajj3085:
david.wendelken:
Alternatively, you could modify your database design and store sent quotations in one set of data tables and quotations in-process in another.  When a quotation is sent, it copies the current product prices and moves itself to the sent quotation tables (and removes itself from the unsent ones.)  The in-process quotation line item would not actually have a line item price, as it would look up the product info for the base line item price.  This would avoid the entire update the unsent quotation line item issue entirely.  It also has the advantage of avoiding fragmentation of your quotation table storage area.  Sent quotations shouldn't (normally) be changed, whereas unsent quotations might be changed a lot (depending upon your business) as you dicker with the customer over requirements.


Well, I'm not a fan of denormalization to begin with.  Keep in mind that I also have Orders and Invoices, whose line item elements function identically to the Quote.  I'll also very likely have an RMA document in the next phase, although i'm not clear if it will be function similar to the existing documents.  So with this method, i'm up to six identical tables.

You misunderstood.  You would not have identical tables with this approach.  The line item tables for unsent quotations would not contain the current product price.  They would be different (only) in that regard.

ajj3085:

david.wendelken:
Every time you update a quotation line item price, the denormalized total needs to be updated.  For this purpose, sent/unsent is of no consequence.  (If you allow a change to quotation line item then you have to update the totals, period.)  This is a pure database issue and a trigger would be excellent here.   No matter how many different objects mess with quotation line item data, the database can keep them synchronized.


That would be a fine solution as well, except that product price changes won't be reflected, which was a problem when this solution was originally suggested.  Its not so much of a problem if the user's have to open the quote for editing to see any data, but that would be a poor user experience I would think.  If they have a list of quotes (which are read only objects) it would be ideal to give them summary data there (which is actually why I posted this; the editable business objects load everything and thus always get the correct data.. so its only an issue for my readonly summary objects).

Nope, not true.  Product prices would be reflected in unsent quotations!

A change to a product price (via the product business object) should cause an update to all unsent quotation line items for that product. 

update quotation_line_item
set product_unit_price = @new_product_unit_price
where product_id = @product_id
    and quotation_id in (select quotation_id 
                                     from quotation
                                    where quotation_status = 'Not Sent'
                                 )

The database trigger on quotation_line_item would be responsible for updating the quotation total_price column value for the changed quotations.


 

ajj3085 replied on Wednesday, October 25, 2006

david.wendelken:
You misunderstood.  You would not have identical tables with this approach.  The line item tables for unsent quotations would not contain the current product price.  They would be different (only) in that regard.


Well, that still violates normalization, and I would end up with six tables still.

david.wendelken:
Nope, not true.  Product prices would be reflected in unsent quotations!

A change to a product price (via the product business object) should cause an update to all unsent quotation line items for that product. 

update quotation_line_item
set product_unit_price = @new_product_unit_price
where product_id = @product_id
    and quotation_id in (select quotation_id 
                                     from quotation
                                    where quotation_status = 'Not Sent'
                                 )

The database trigger on quotation_line_item would be responsible for updating the quotation total_price column value for the changed quotations.


Ahh, I see.  That would work, although I would have concerns about performance.  There is functionality that allows prices for products to be updated in one transaction.. and I know LineItems will be a table that will quickly accumlate many rows. 

I'm leaning toward your first suggestion about just letting the business objects handle it.  That way I avoid any denomalization or trigger problems (at one employer we tried them, and had them cause so many hard to find problems we ended up deleting all the triggers and making the updates happen by other means). 

This is my current though; have a DocumentListSummary object, which would load all the line items for all the documents in the List (this would happen as part of the DocumentList fetch operation).  Loading of this object would dynamically build the summary information, and the DocumentInfo classes woudl delegate to this class to find out the summary information.

Only two database hits, and the processing should be pretty quick.  The db remains clean as well. 

Andy

david.wendelken replied on Wednesday, October 25, 2006

One of the advantages of the "un-sent" line item table is that it wouldn't get really large - as quotes were sent it would shrink back. :)

Disadvantages are as you noted, multiple similar tables.

Computer science is the science of giving away some features you want in order to get other features you want more strongly. :)

 

Copyright (c) Marimer LLC