Login if you already have an account



Create a new account. join us!



Need some help? Check out our forum.

July 15, 2014 Dave Wells

Many-to-Many Relationships & NoSQL? Problem solved.

I’m an engineer and not usually given to making sweeping statements like, “we’ve solved the many-to-many relationships problem for NoSQL;” but in this case, I hope you’ll agree, it’s merited.

In short, with FatFractal we have created a simple and efficient way for you to code extremely-complex relationships between models.

The many-to-many relationships problem is classic and one which SQL solved long ago with joins. However, there are a couple of well-known issues: 1) a very large number of join tables can become so complicated that you’re better off re-architecting your data model; and 2) in the cloud, even with a good sharding strategy, you can run into scaling problems when things go big.

Developers do one of two things in NoSQL to relate objects to each other: either add a field for each object containing referents to the other or add a third object containing the relationship between the two (can you say “join table?”). These options are talked about in lots of places, see for example Google’s take and Kinvey’s and Parse’s. In each of these main approaches, the clientside, code complexity increases for every new relationship; if you add lots of different relationships, across lots of objects, your complexity grows geometrically to failure.

FatFractal’s solution is a backend datatype called ‘GRABBAG’. Simply put, a grab bag is a set of references to any number of other objects – potentially, any collection and potentially, any objecttype. Grab bags are not part of your object model: they are maintained separately by your app’s backend and we give you methods to CRUD them from clients.

Enough talk – let’s do code.

To show you how straightforward we’ve made these many-to-many relationships, we’ll create a ‘Movie’ object and a ‘Theater’ object. A movie can be showing in many theaters and a theater can be showing many movies. First, we need a method to allow movies to be added to theaters.

We’ll model up the data using our simple, markup language, FatFractal Definition Language (FFDL, “fiddle”) saved to a config file (there are other options for modeling—but FFDL is particularly easy to follow):

CREATE OBJECTTYPE Theater (theaterName STRING, location GEOLOCATION, movies GRABBAG)

Here’s a set of Objective-C interfaces:

@interface Movie : NSObject
@property (strong, nonatomic) NSString *movieName;
- (void) getTheaters:(id )delegate;
@interface Theater : NSObject
@property (strong, nonatomic) NSString *theaterName;
@property (strong, nonatomic) FFGeoLocation *location;
- (void) getMovies:(id )delegate;
- (NSError *) addMovie:(Movie *)movie;
@protocol SimpleLoadProtocol
- (void) didLoad:(id)data;
- (void) didFailLoadWithError:(NSError *)err;

Here’s the Objective-C code required to implement Theater’s addMovie and getMovies methods:

- (NSError *) addMovie:(Movie *)movie
NSError *err;
[[FatFractal main] grabBagAdd:movie to:self grabBagName:@"movies" error:&err];
return err;
- (void) getMovies:(id )delegate
FatFractal *ff = [FatFractal main];
__block id theDelegate = delegate;
[ff grabBagGetAllForObj:self grabBagName:@"movies" onComplete:^(NSError *err, id movieList, NSHTTPURLResponse *theResponse) {
if (err)
[theDelegate didFailLoadWithError:err];
[theDelegate didLoad:movieList];

Yes, that really is it—just add a ‘movies’ member of backend datatype GRABBAG to the Theater OBJECTTYPE and everything else is taken care of for you. Happily, even though the grab bag is defined in your Theater OBJECTTYPE, as far as your app and its backend are concerned, both the model and the data are held separate-and-apart from the Theater objects.

Relationships can come and go, while your clientside and serverside object models stay pure.

Some Test Data (showing relationships)
BalboaThe Artist, Balboa
CastroMelancholia, Tree of Life, The Trip, A Separation
LumiereBeginners, A Separation, Melancholia
Some Snippets:

Add a movie “The Future” to show at Theater Balboa:

Movie * theFuture = [[Movie alloc] init]; // create new movie
[theFuture setMovieName:@"The Future"]; // its name is “The Future”
[ff createObj:theFuture atUri:@"/Movies"]; // save it!
Theater * balboa = [ff getObjFromUri:@"/Theaters/(theaterName eq 'Balboa')"];
[balboa addMovie:theFuture]; // add the movie

All that the addMovie method is doing under the hood is:

[[FatFractal main] grabBagAdd:theFuture to:balboa grabBagName:@"movies" error:&err];

Query: Show all movies at the Lumiere theater:

NSArray * moviesShowingAtLumiere = [ff getArrayFromUri:@"/Theaters/(theaterName eq 'Lumiere')/movies"];

What about getting the list of theaters that a movie is showing in?

For every “forward” relationship represented in a FatFractal object, whether by a REFERENCE or a GRABBAG, FatFractal automatically creates and maintains the “inverse relationship” from the referred-to object. Those inverse relationships are available through the system-maintained ‘Back References’ GRABBAG. Every object’s back references are kept by the system grab bag. And why not? After all, grab bags are not part of your clientside data model or your serverside data model; so there’s no impact on you or your code. Create as many grab bags (i.e. relationships between objects) as you like. Your code will stay elegant and simple.

Without any more modeling required, here’s a method to get the Theater objects where a movie(s) is playing:

- (void) getTheaters:(id)delegate
FatFractal *ff = [FatFractal main];
__block id theDelegate = delegate;
[ff grabBagGetAllForObj:self grabBagName:@"BackReferences.Theaters.movies" onComplete:^(NSError *err, id theaterList, NSHTTPURLResponse *theResponse) {
if (err)
[theDelegate didFailLoadWithError:err];
[theDelegate didLoad:theaterList];

Query: Show me all the theaters where “A Separation” is playing.

NSArray * theatersShowingASeparation = [ff getArrayFromUri:@"/Movies/(movieName eq 'A Separation')/BackReferences.Theaters.movies"];

The syntax may seem a bit strange but it means,”start with the /Movies collection and select those Movie objects with movieName equal to “A Separation.” From that set, give me the list of objects in the /Theaters collection whose ‘movies’ member references the previous list of objects (in this case, a single Movie object, ‘A Separation’). In plain English, give me all the Theater objects pointing to the Movie object ‘A Separation.’

Here’s the real payoff: let’s say we want to add relationships between Movie objects and movie stars. Obviously, movies have multiple stars and stars can be in multiple movies. Let’s add a movie star model and set up yet-another, many-to-many relationship:

CREATE OBJECTTYPE Theater (theaterName STRING, location GEOLOCATION, movies GRABBAG /Movies)
CREATE OBJECTTYPE Movie (movieName STRING, stars GRABBAG /Stars)
Data set: showing relationships
Movies (Stars)
BalboaThe Artist (Jean Dujardin, Bérénice Bejo), Babel (Brad Pitt, Cate Blanchett, Gael García Bernal)
CastroMelancholia (Kirsten Dunst, Charlotte Gainsbourg), Tree of Life (Brad Pitt, Sean Penn, Jessica Chastain), The Trip (Steve Coogan, Rob Brydon), A Separation (Peyman Moadi, Leila Hatami)
LumiereBeginners (Ewan McGregor, Christopher Plummer), A Separation (Peyman Moadi, Leila Hatami), Melancholia (Kirsten Dunst, Charlotte Gainsbourg)

Simply adding ‘stars GRABBAG /Stars’ to the Movie objecttype establishes the new relationship. No additional tables or code complexity or future code maintenance headaches. And you can do some pretty cool queries!

Query: Get all the stars in all of the movies showing at the Lumiere.

NSArray * starringInMoviesShowingAtLumiere = [ff getArrayFromUri:@"/Theaters/(theaterName eq 'Lumiere')/movies/()/stars"];

Query: Show me all the movies that Brad Pitt stars in.

NSArray * bradPittMovies = [ff getArrayFromUri:@"/Stars/(movieStarName eq 'Brad Pitt')/()/BackReferences.Movies.stars"];

Query: Show me all the theaters that are showing movies that Brad Pitt stars in.

NSArray * theatersShowingBradPittMovies = [ff getArrayFromUri:@"/Stars/(movieStarName eq 'Brad Pitt')/BackReferences.Movies.stars/()/BackReferences.Theaters.movies"];

(note: that back references syntax can get unwieldy so we have a way to “alias” those references.)

Now, just for fun (I stuck in a GEOLOCATION member in the object model just for this!), let’s find out which theaters are showing “Avengers Assemble” within 50 kilometers of the town of Nairn in Northern Scotland where I live:

[ff getArrayFromUri:@"/Movies/(movieName eq 'Avengers Assemble')/theaters/(distance (location, [57.5833, 3.8667]) lte 50000)"];

No more geometrically-expanding “joins” with all maintenance nightmares that such complexity entails. FatFractal offers a much simpler solution…and 50% is handed to you for free with back references! Model the relationships from theaters to movies and from movies to stars and the system’s back references let you traverse the datagraph in reverse: from stars to movies to theaters.

Create relationships. Save time. Stay sane…and have fun!