Sunday, June 1, 2014

Big Data, Normalization & Analytics



May Post @All Analytics

What you need to know for the purposes of this discussion is that tables that bundle multiple entity classes have certain drawbacks. Normalization is a design repair procedure that unbundles the classes -- the columns representing attributes pertaining to each class -- each into its own table. This is possible if and only if there is no data lost or made spurious in the process -- that is, when a bundling of table A is mathematically equivalent to the joins of its unbundled projection tables B and C.

Read it all. (And please comment there, not here)



 



Tuesday, May 27, 2014

Weekly Update





1. Quote of the Week
Codd's relational model is based on set theory, and set theory simply doesn't work for database systems. It can't, for example, model a gum ball machine. Gum balls, you see, have only one attribute, which is color (gum balls don't have names, serial numbers, bar codes, or URLs). If you put 200 gum balls in a gum ball machine, the gum ball machines contains 200 gum balls. If you try to put 200 gum balls in a gum ball relation, you get a relation of 5 gum balls (the number of colors) and 195 duplicate errors. If you then take 5 gum balls out of the gum ball machine, it still contains 195 gum balls. If you take 5 gum balls out of the gum ball relation, it goes empty. --Jim Starkey, LinkedIn.com

2. To Laugh or Cry?
How to store and document large data models

3. Online

4. Interesting elsewhere
Software engineers think they're amazingly great 

5. And now for something completely different
God keeps missing.




Sunday, May 18, 2014

For Codd's Sake -- UPDATED




UPDATE: Correction on 6/8/14


This is a response to comments by a reader on one of my posts.
L: I realize that you have taken much further what Codd wrote on the first page of his 1970 paper but it's still remarkable how many people in the data business are not able to refer to, let alone talk productively, about his "natural structure of data". And many treat RT as a fait accompli when it is still evolving, not to mention those who, as you've pointed out many times, treat SQL gizmos such as outer join as if they come from RT when they don't.

Sunday, May 11, 2014

Weekly Update




1. I will give the following presentation

Big Data, Analytics and Normalization
"Big Data may offer analytical insights, but with almost certainty will produce really big lies from 100% correct data", particularly when data are from external sources. This presentation will demonstrate
  • Why and how
  • How to protect yourself
Wednesday, 5/14, 7:00pm
Microsoft San Francisco office
835 Market St.

For more information contact MGinnebaugh@designmind.com.


2. Quote of the Week
Q: How do we do data modeling in NoSQL DB and Big Data??? 
A: Define the schema hierarchically so that the tables in the schema including ER form a forest using a parent relationship i.e. each table has at most one parent key. Now the data retrieval and storage is done using these parent or ancestor keys. Look for google datastore documentation for more details. --LinkedIn.com

3. To Laugh or Cry? and Online

David McGoveran's comments posted last week are a response to the following LinkedIn exchange initiated by Jim Starkey of Rdb and Interbase fame:
Is the Relational Data Model Spent?
Given who Jim is, my instinct is to cry rather than laugh. This is also the Online item, as I participated in the exchange. Jim did not respond publicly to my challenges and claimed in private that I was trolling. You decide, but if I am a troll, so is David.


4. Interesting elsewhere
Do graph databases deprecate relational databases?
H/t Erwin Smout.


5. And Now for Something Completely Different
No comment.



Sunday, April 27, 2014

UPDATE 2: David McGoveran: Comments on Jim Starkey's "Is the Relational Data Model Spent?"




UPDATE 1: I have added Jim Starkey's reply to David's initial response (with my brief comments) and David's reply to it below.

UPDATE 2: I have made a few minor corrections and fixed end-note formatting problems.


David McGoveran's First Response  
© 2014 David McGoveran – All Rights Reserved
Jim Starkey's opinions in Is the relational model spent?, a LinkedIn exchange he initiated, reflect those of many professionals who have used and even developed SQL DBMSs and their predecessors. While the concerns with so-called "commercial relational database systems" expressed by Jim are valid, they have nothing to do with the relational (data) model. They are the result of DBMS implementations by those who borrowed something from the relational model, but never understood it and so did not know how to take advantage of it to solve application problems.

Weekly Update - UPDATED




1. I will give the following presentation

Big Data, Analytics and Normalization
"Big Data may offer analytical insights, but with almost certainty will produce really big lies from 100% correct data", particularly when data are from external sources. This presentation will demonstrate
  • Why and how
  • How to protect yourself
Wednesday, 5/14, 7:00pm
Microsoft San Francisco office
835 Market St., 7th Floor
San Francisco

For more information contact MGinnebaugh@designmind.com.


2. My April column @All Analytics:
Missing Data, Databases & Analytics

3. Quote of the Week
Q: Is it necessary to follow standards during SQL programming? 
 
A: Standards and Best Practices usually come from common sense. I want to point out that it is God given potential which one must realise and be conscious to utilize it for His glory. --LinkedIn.com

4. To Laugh or Cry?
A data model of the SAP Bill of Material Explosion tables

5. Online 


6. Interesting elsewhere
Big Data, Little Happiness
(requires free registration)


7. And now for something completely different
The Death Of Expertise
Today being the anniversary of the Holocaust, I decided to add the following:
Berlusconi's holocaust jibe provokes German outrage
The irony of Italians badmouthing the Germans about the extermination of Jews. But this time the former spoke the truth:the latter cannot have it both ways.




Sunday, April 20, 2014

Forward to the Past: From Codd to SQL to NoSQL




As told by C. J. Date, sometime shortly after the introduction of SQL DBMS's in the industry, when non-relational products e.g. hierarchic and network reigned and the relational idea was a very hard sell, he and Michael Stonebraker (the author of Ingres and at the time a professor of Computer Science at University of California Berkeley) participated in a panel at a technical conference. The following is the (praphrased) exchange between them:
CJD: The reality is that most practitioners are too set in their non-relational ways and we cannot expect them to understand and appreciate the relational model. Rather, we must focus on the young generation of practitioners, who learn the relational model at university.

MS: Chris, you don't understand. I am teaching those youths: they were not around when we struggled with the huge problems of the pre-relational systems and they are reinventing all of them!

View My Stats