X
X

Login

Login if you already have an account

LOGIN

Register

Create a new account. join us!

REGISTER

Support

Need some help? Check out our forum.

FORUM

Many-to-Many Relationships & NoSQL? Problem solved.

I’m an engineer and not usually given to making sweeping statements like, “we’ve solved the many-to-many relationships problem for NoSQL;” but in this case, I hope you’ll agree, it’s merited.

I’m an engineer and not usually given to making sweeping statements like, “we’ve solved the many-to-many relationships problem for NoSQL;” but in this case, I hope you’ll agree, it’s merited.

In short, with FatFractal we have created a simple and efficient way for you to code extremely-complex relationships between models.

The many-to-many relationships problem is classic and one which SQL solved long ago with joins. However, there are a couple of well-known issues: 1) a very large number of join tables can become so complicated that you’re better off re-architecting your data model; and 2) in the cloud, even with a good sharding strategy, you can run into scaling problems when things go big.

Developers do one of two things in NoSQL to relate objects to each other: either add a field for each object containing referents to the other or add a third object containing the relationship between the two (can you say “join table?”). These options are talked about in lots of places, see for example Google’s take and Kinvey’s and Parse’s. In each of these main approaches, the clientside, code complexity increases for every new relationship; if you add lots of different relationships, across lots of objects, your complexity grows geometrically to failure.

FatFractal’s solution is a backend datatype called ‘GRABBAG’. Simply put, a grab bag is a set of references to any number of other objects – potentially, any collection and potentially, any objecttype. Grab bags are not part of your object model: they are maintained separately by your app’s backend and we give you methods to CRUD them from clients.

Enough talk – let’s do code.

To show you how straightforward we’ve made these many-to-many relationships, we’ll create a ‘Movie’ object and a ‘Theater’ object. A movie can be showing in many theaters and a theater can be showing many movies. First, we need a method to allow movies to be added to theaters.

We’ll model up the data using our simple, markup language, FatFractal Definition Language (FFDL, “fiddle”) saved to a config file (there are other options for modeling—but FFDL is particularly easy to follow):

CREATE OBJECTTYPE Theater (theaterName STRING, location GEOLOCATION, movies GRABBAG)
CREATE OBJECTTYPE Movie (movieName STRING)
CREATE COLLECTION /Theaters OBJECTTYPE Theater
CREATE COLLECTION /Movies OBJECTTYPE Movie

Here’s a set of Objective-C interfaces:

@interface Movie : NSObject
@property (strong, nonatomic) NSString *movieName;
- (void) getTheaters:(id )delegate;
@end

@interface Theater : NSObject
@property (strong, nonatomic) NSString *theaterName;
@property (strong, nonatomic) FFGeoLocation *location;
- (void) getMovies:(id )delegate;
- (NSError *) addMovie:(Movie *)movie;
@end

@protocol SimpleLoadProtocol
- (void) didLoad:(id)data;
- (void) didFailLoadWithError:(NSError *)err;
@end

Here’s the Objective-C code required to implement Theater’s addMovie and getMovies methods:

- (NSError *) addMovie:(Movie *)movie
{
NSError *err;
[[FatFractal main] grabBagAdd:movie to:self grabBagName:@"movies" error:&err];
return err;
}

- (void) getMovies:(id )delegate
{
FatFractal *ff = [FatFractal main];
__block id theDelegate = delegate;
[ff grabBagGetAllForObj:self grabBagName:@"movies" onComplete:^(NSError *err, id movieList, NSHTTPURLResponse *theResponse) {
if (err)
[theDelegate didFailLoadWithError:err];
else
[theDelegate didLoad:movieList];
}];
}

Yes, that really is it—just add a ‘movies’ member of backend datatype GRABBAG to the Theater OBJECTTYPE and everything else is taken care of for you. Happily, even though the grab bag is defined in your Theater OBJECTTYPE, as far as your app and its backend are concerned, both the model and the data are held separate-and-apart from the Theater objects.

Relationships can come and go, while your clientside and serverside object models stay pure.

Some Test Data (showing relationships)
Theaters
Movies
BalboaThe Artist, Balboa
CastroMelancholia, Tree of Life, The Trip, A Separation
LumiereBeginners, A Separation, Melancholia
Some Snippets:

Add a movie “The Future” to show at Theater Balboa:

Movie * theFuture = [[Movie alloc] init]; // create new movie
[theFuture setMovieName:@"The Future"]; // its name is “The Future”
[ff createObj:theFuture atUri:@"/Movies"]; // save it!
Theater * balboa = [ff getObjFromUri:@"/Theaters/(theaterName eq 'Balboa')"];
[balboa addMovie:theFuture]; // add the movie

All that the addMovie method is doing under the hood is:

[[FatFractal main] grabBagAdd:theFuture to:balboa grabBagName:@"movies" error:&err];

Query: Show all movies at the Lumiere theater:

NSArray * moviesShowingAtLumiere = [ff getArrayFromUri:@"/Theaters/(theaterName eq 'Lumiere')/movies"];

What about getting the list of theaters that a movie is showing in?

For every “forward” relationship represented in a FatFractal object, whether by a REFERENCE or a GRABBAG, FatFractal automatically creates and maintains the “inverse relationship” from the referred-to object. Those inverse relationships are available through the system-maintained ‘Back References’ GRABBAG. Every object’s back references are kept by the system grab bag. And why not? After all, grab bags are not part of your clientside data model or your serverside data model; so there’s no impact on you or your code. Create as many grab bags (i.e. relationships between objects) as you like. Your code will stay elegant and simple.

Without any more modeling required, here’s a method to get the Theater objects where a movie(s) is playing:

- (void) getTheaters:(id)delegate
{
FatFractal *ff = [FatFractal main];
__block id theDelegate = delegate;
[ff grabBagGetAllForObj:self grabBagName:@"BackReferences.Theaters.movies" onComplete:^(NSError *err, id theaterList, NSHTTPURLResponse *theResponse) {
if (err)
[theDelegate didFailLoadWithError:err];
else
[theDelegate didLoad:theaterList];
}];
}

Query: Show me all the theaters where “A Separation” is playing.

NSArray * theatersShowingASeparation = [ff getArrayFromUri:@"/Movies/(movieName eq 'A Separation')/BackReferences.Theaters.movies"];

The syntax may seem a bit strange but it means,”start with the /Movies collection and select those Movie objects with movieName equal to “A Separation.” From that set, give me the list of objects in the /Theaters collection whose ‘movies’ member references the previous list of objects (in this case, a single Movie object, ‘A Separation’). In plain English, give me all the Theater objects pointing to the Movie object ‘A Separation.’

Here’s the real payoff: let’s say we want to add relationships between Movie objects and movie stars. Obviously, movies have multiple stars and stars can be in multiple movies. Let’s add a movie star model and set up yet-another, many-to-many relationship:

CREATE OBJECTTYPE Theater (theaterName STRING, location GEOLOCATION, movies GRABBAG /Movies)
CREATE OBJECTTYPE Movie (movieName STRING, stars GRABBAG /Stars)
CREATE OBJECTTYPE Star (movieStarName STRING)

CREATE COLLECTION /Stars OBJECTTYPE Star
CREATE COLLECTION /Theaters OBJECTTYPE Theater
CREATE COLLECTION /Movies OBJECTTYPE Movie

Data set: showing relationships
Theaters
Movies (Stars)
BalboaThe Artist (Jean Dujardin, Bérénice Bejo), Babel (Brad Pitt, Cate Blanchett, Gael García Bernal)
CastroMelancholia (Kirsten Dunst, Charlotte Gainsbourg), Tree of Life (Brad Pitt, Sean Penn, Jessica Chastain), The Trip (Steve Coogan, Rob Brydon), A Separation (Peyman Moadi, Leila Hatami)
LumiereBeginners (Ewan McGregor, Christopher Plummer), A Separation (Peyman Moadi, Leila Hatami), Melancholia (Kirsten Dunst, Charlotte Gainsbourg)

Simply adding ‘stars GRABBAG /Stars’ to the Movie objecttype establishes the new relationship. No additional tables or code complexity or future code maintenance headaches. And you can do some pretty cool queries!

Query: Get all the stars in all of the movies showing at the Lumiere.

NSArray * starringInMoviesShowingAtLumiere = [ff getArrayFromUri:@"/Theaters/(theaterName eq 'Lumiere')/movies/()/stars"];

Query: Show me all the movies that Brad Pitt stars in.

NSArray * bradPittMovies = [ff getArrayFromUri:@"/Stars/(movieStarName eq 'Brad Pitt')/()/BackReferences.Movies.stars"];

Query: Show me all the theaters that are showing movies that Brad Pitt stars in.

NSArray * theatersShowingBradPittMovies = [ff getArrayFromUri:@"/Stars/(movieStarName eq 'Brad Pitt')/BackReferences.Movies.stars/()/BackReferences.Theaters.movies"];

(note: that back references syntax can get unwieldy so we have a way to “alias” those references.)

Now, just for fun (I stuck in a GEOLOCATION member in the object model just for this!), let’s find out which theaters are showing “Avengers Assemble” within 50 kilometers of the town of Nairn in Northern Scotland where I live:

[ff getArrayFromUri:@"/Movies/(movieName eq 'Avengers Assemble')/theaters/(distance (location, [57.5833, 3.8667]) lte 50000)"];

No more geometrically-expanding “joins” with all maintenance nightmares that such complexity entails. FatFractal offers a much simpler solution…and 50% is handed to you for free with back references! Model the relationships from theaters to movies and from movies to stars and the system’s back references let you traverse the datagraph in reverse: from stars to movies to theaters.

Create relationships. Save time. Stay sane…and have fun!

FatFractal Emerges from Stealth Mode

Big day for the FatFractal team: public announcement and some great press (read the press release below or click here).

Yes, customer happiness and successful deployments are the most important thing. But it doesn’t hurt to be talked about by knowledgeable press, especially when the talk is good. Thanks to all our supporters out there and we look forward to working with you in a more….public….way.

If you’re at TechCrunch this week in San Francisco, stop by and see us. We’ll be in Startup Alley on Tuesday.

[BTW a few of you mentioned you had trouble registering this morning: minor bug in last night's registration journey push has been fixed. To the twenty of you that had problems, our apologies.]

FULL TEXT OF THE PRESS RELEASE

FatFractal Emerges from Stealth Mode with End-to-End Platform for Developing Applications with Cloud Backends on Mobile Phones, Browsers or Any Connected Device

Engine-Based Platform Combines the Power of an Engine-based PaaS with Easy-to-Use and Cost Effective BaaS Services

September 10, 2012 — FatFractal, an end-to-end solution for developing applications with a cloud backend on any connected device, announced today the launch of the company and platform. The company’s mission is to significantly reduce the time and cost to create web and device-based apps with cloud backends. FatFractal makes it easy to create and deploy a backend, offering never-before-seen, patent-pending features.

As developers themselves, the founders of the company recognized that creating applications with a cloud backend was far too hard. Since the founders couldn’t find a solution for creating apps with the optimal application architecture for the cloud, they built one (FatFractal Platform Architecture). Before, developers could spend half their effort on busy-work, configuring an infrastructure, setting up a software stack, managing scalability and optimizing performance and security. FatFractal eliminates all that busy work as well as the need to write boilerplate code. With FatFractal, developers can focus on creating engaging and dynamic apps.

“The Platform as a Service (PaaS) players like Google and Microsoft offer middleware services that tie developers to their infrastructures. Our engine-based PaaS is highly scalable, lightweight and not locked in to any infrastructure,” said Kevin Nickels, President of FatFractal. “The Backend as a Service (BaaS) companies provide an instant datastore but the backends they provide are mostly black boxes to application developers. FatFractal gives developers easy access and control over their backends offering productivity improvements of up to 70 percent for applications with complex backend requirements, significantly improving time to market and lowering the cost of development.”

To build an application for mobile, the web or any connected device, a developer codes up the client application and starts persisting data: the backend learns the object model dynamically and it all just works. Developers also have the option of creating schemas to validate their objects using a simple description language called FatFractal Definition Language (FFDL, “fiddle”). In addition to an “instant backend,” the FatFractal platform offers many BaaS features “out of the box,” including the following:

  • Every app gets an instant, customizable REST API
  • The FatFractal serializer/deserializer sends object information over the wire as JSON/HTTP
  • User and Group objects are provided with inherent authentication and methods to login and logout
  • FatFractal provides lightweight services such as geolocation, push, email, payment, social integration using OAuth 2.0
  • A console and data browser for quickly viewing, administrating and managing applications, domains and data

“From a programmer’s point of view, my platform should work for me, not force me to work for it,” stated Gary Casey, founder and software engineer at FatFractal. “We built FatFractal so that developers will constantly be delighted: they will find themselves creating efficient, elegant apps with 50 percent fewer lines of code, certainly no boilerplate code. Our SDKs are designed to fit in with native programming environments whether the coder is using Objective-C, Java or HTML5/JS or even serversside JavaScript— you can always use native objects, classes, methods and tools exactly as you are used to doing.”

Unlike any other application development platform on the market, FatFractal provides features that developers crave to create applications with cloud backends easily, efficiently and cost-effectively:

  • Enterprise-grade Security: From authentication to sandboxing to encryption at rest, every part of the platform is built to ensure security for both the application and users’ data. FatFractal provides declarative security so developers can set default permissions on collections for specified users and groups, as well as dynamic: at run-time, programmers can change permissions at the collection- and even the object-level.
  • Datagraph: FatFractal has solved a vexing problem for NoSQL datastores involving large numbers of many-to-many relationships by introducing Grab Bags and Back References. Using SQL databases to manage lots of relationships across lots of objects can result in insurmountable complexity as more and more join tables get created. RDBMS’s scale poorly in a cloud context. FatFractal solves that problem for developers for the first time. (Many-to-Many Relationships & NoSQL? Problem solved.) Developers can now model intricate social-networks and complex datagraphs without giving up performance or scalability.
  • Events: Each CRUD action generates an event which can be trapped and sent to any handler code. This unique feature gives developers flexibility and control. Events can happen before a CRUD action (PRE event), after a CRUD action (POST event, sync), asynchronously (ASYNC) or scheduled (CRON). Event handlers can be cascaded as well, allowing the app’s backend to execute any number of handlers on a CRUD action—it’s easy to set up and extremely powerful. Handlers can be written in any language supported by FatFractal, for example serverside JavaScript.
  • Custom Code: Developers can extend their application’s API with custom code. These server extensions are attached to URIs, allowing developers to take advantage of their apps’ backends to run business logic. As with event handlers, custom code can be written in serverside JavaScript.
  • Complex Queries: Developers can create complex, nested queries without creating special query objects or methods. Object permissions are always respected and because of our unique datagraph, including grab bags and back references, developers can traverse their object models forwards and backwards simply by using queries and without writing any of their own code for that purpose.
  • Language Modules: Given FatFractal is a fully-formed PaaS, language modules such as Ruby are provided for developers and the open module standard means third parties can create any language modules they might wish as well.
  • Local Development: The FatFractal engine can run on any infrastructure and any operating system; in fact, the engine can run on anything that supports the JVM. Developers create their apps on a local FatFractal Engine, using all the tools, methods and testing techniques they already know. Since the Engine is exactly the same locally as in the cloud, developers can deploy to the cloud with a single command and be confident that their app will work exactly the same.

About FatFractal

FatFractal offers an applications development platform that combines the power of an engine-based PaaS with the elegance and simplicity of a BaaS, so developers can quickly go from code to cloud. Developers build native apps for mobile, web and any connected device the way they should be built: quickly and efficiently, leveraging native code on the clientside and harnessing the power of the backend.  Creating engaging apps with a cloud backend takes less time, results in elegant code and developers can deploy them, scale them and manage them at a far lower cost than ever before. For more information, see http://www.fatfractal.com.

Contact