software solutions / project leadership / agile coaching and training

Moving past the monolith, Part 2 – Separating your data

Posted on November 4, 2017

This is part of a series of posts about Moving Past The Monolith. To start at the beginning, click here.


Separating your business logic code into modules is one thing, but in many cases, that is the easy part. The tough part is the data.

In most large applications, you can probably think of a few database tables that are at the core of the application, have many other tables linking to it, and are pretty much used everywhere.

While you probably won’t be able to achieve total separation of concerns in your data, there are things you do to separate things as much as you can. You will probably always have some shared tables, but knowing that you’re only sharing some of them is better than having to assuming that you’re sharing all of them.

Schemas

One simple approach is to create separate schemas in your database (assuming you’re in a database like SQL Server or Oracle that supports multiple schemas within a database). Schemas are a good way to keep things separate – you can assign permissions at the schema level, and they are a good indication of which code module owns a database table. Since the schemas are all in the same database, you can write queries across schemas if you want to (as long as you acknowledge that you’re introducing more coupling by doing so).

What about the shared things?

What do you do with the core tables that need to be shared across all modules? You still have options here.

  • You could acknowledge that every module needs to link to a given table, and just live with it
  • You could have one shared table that contains the primary key and shared data points, and then each schema has its own table which has columns that only it cares about
  • Each schema could have its own version of the shared data, with some batch process keeping the data from a master table in sync with copies of the data in other schemas

Here are some questions that you should ask:

  • Can you keep this table in a schema for a given module, or do you need the same data in multiple modules?
  • Could you easily move a schema and its tables into a completely separate database if you wanted to? What would happen if you did, and how hard would it be?
  • If you have multiple versions of the same data in different schemas, how will you keep them in sync? Does the benefit of having the separation outweigh the work it will take to keep the data in sync?
  • Does all of your data need to be completely up to date in real time, or can certain data be updated nightly or at some interval with a batch job?

There are a lot of trade-offs to consider here, and I imagine that most people will have some shared tables that are used throughout the application. That is fine, but what I think is important is that you consider the trade-offs that come with a shared table, and if you choose to accept that because it’s worth it, then go with it. But the default should be to try and keep table separated, and move them together intentionally.


Read the next post in this series, Think about how you share data.

No Comments »

No comments yet.

Leave a comment





SERVICES
SOFTWARE SOLUTIONS
I have over 15 years of software development experience on several different platforms (.NET, Ruby, JavaScript, SQL Server, and more). I recognize that software is expensive, so I'm always trying to find ways to speed up the software development process, but at the same time remembering that high quality is essential to building software that stands the test of time.
PROJECT LEADERSHIP
I have experience leading and architecting large Agile software projects and coordinating all aspects of a project's lifecycle. Whether you're looking for technical expertise or someone to lead all aspects of an Agile project, I have proven experience from multiple projects in different environments that can help make your project a success.
PROCESS COACHING
Every team and every situation is different, and I believe that processes and tools should be applied with common sense. I've spent the last 10+ years working on projects using Agile and Lean concepts in many different environments, both in leadership roles and as a practitioner doing the work. I can help you develop a process that works best in your organization, not just apply a prescriptive process.
Have any questions? Contact me for more information.
PRESENTATIONS
From Stir Trek 2017
Iteration Management - Your Key to Predictable Delivery
From Stir Trek 2016 and QA or the Highway 2015
From CodeMash 2016, QA or the Highway 2014, Stir Trek 2012
The Business of You: 10 Steps For Running Your Career Like a Business
From CodeMash 2015, Stir Trek 2014, CONDG 2012
From Stir Trek 2013, DogFoodCon 2013
(presented with Brandon Childers, Chris Hoover, Laurel Odronic, and Lan Bloch from IGS Energy) from Path to Agility 2012
From CodeMash 2012 and 2013
(presented with Paul Bahler and Kevin Chivington from IGS Energy)
From CodeMash 2011
An idea of how to make JavaScript testable, presented at Stir Trek 2011. The world of JavaScript frameworks has changed greatly since then, but I still agree with the concepts.
A description of how test-driven development works along with some hands-on examples.
From CodeMash 2010
From CodeMash 2010