Section 5.2 Data Modeling and Entity-Relationship Diagrams
Fortunately there is an extensive area of study and practice called “data modeling” that provides theories, strategies, and tools to help with the data scientist’s goal of following the data. These ideas started in earnest in the 1970s with the introduction by computer scientist Ed Yourdon of a methodology called Data Flow Diagrams. A more contemporary approach, that is strongly linked with the practice of creating relational databases, is called the entity-relationship model. Professionals using this model develop Entity-Relationship Diagrams (ERDs) that describe the structure and movement of data in a system.
Entity-relationship modeling occurs at different levels ranging from an abstract conceptual level to a physical storage level. At the conceptual level an entity is an object or thing, usually something in the real world. In the doctor’s office example, one important “object” is the patient. Another entity is the doctor. The patient and the doctor are linked by a relationship: in modern health care lingo this is the “provider” relationship. If the patient is Mr. X and the doctor is Dr. Y, the provider relationship provides a bidirectional link:
-
Dr. Y is the provider for Mr. X
-
Mr. X’s provider is Dr. Y
Naturally there is a range of data that can represent Mr. X: name, address, age, etc. Likewise, there are data that represent Dr. Y: years of experience as a doctor, specialty areas, certifications, licenses. Importantly, there is also a chunk of data that represents the linkage between X and Y, and this is the relationship.
Creating an ERD requires investigating and enumerating all of the entities, such as patients and doctors, as well as all of the relationships that may exist among them. As the beginning of the chapter suggested, this may have to occur across multiple organizations (e.g., the doctor’s office and the insurance company) depending upon the purpose of the information system that is being designed. Eventually, the ERDs must become detailed enough that they can serve as a specification for the physical storage in a database.
In an application area like health care, there are so many choices for different ways of designing the data that it requires some experience and possibly some “art” to create a workable system. Part of the art lies in understanding the users’ current information needs and anticipating how those needs may change in the future. If an organization is redesigning a system, adding to a system, or creating brand new systems, they are doing so in the expectation of a future benefit. This benefit may arise from greater efficiency, reduction of errors/inaccuracies, or the hope of providing a new product or service with the enhanced information capabilities.
Whatever the goal, the data scientist has an important and difficult challenge of taking the methods of today—including paper forms and manual data entry—and imagining the methods of tomorrow. Follow the data!
In the next chapter we look at one of the most common and most useful ways of organizing data, namely in a rectangular structure that has rows and columns. This rectangular arrangement of data appears in spreadsheets and databases that are used for a variety of applications. Understanding how these rows and columns are organized is critical to most tasks in data science.
