X
X

Login

Login if you already have an account

LOGIN

Register

Create a new account. join us!

REGISTER

Support

Need some help? Check out our forum.

FORUM

FYI – Having fun with datagraphs

So most of our sample code on GitHub is a result of a developer’s question that we think has general interest that other developers may find useful. This usually ends up also being a small sample application that will show the result in a simple UI.

But, sometimes we do things just for fun. For today’s example, I decided to code up the excellent descriptions on the NoServer features page

You can access the source code for the sample application here.
You can access the sample application – complete with data, images and all here.

It is interesting that now that it is so easy to create an app with a backend, that one can create the code (couple of hours), wrap it in an app (less than a couple of hours) and then blog about it.

So what this app does:

This application uses a single Objecttype (Person) and a single Collection (Persons) to demonstrate just how powerful a datagraph representation can be, and how super efficient queries can be constructed to “walk” your datagraph to get what you want with a single call in your client.

The FFDL

First, defining the backend in FFDL takes only two lines of code!

CREATE OBJECTTYPE Person (firstName STRING, lastName STRING, gender STRING, mother REFERENCE /Persons, father REFERENCE /Persons, siblings GRABBAG /Persons, picture BYTEARRAY)
CREATE COLLECTION /Persons OBJECTTYPE Person

The REFERENCE associations above (mother and father), these can be considered “one to one” relationships between person objects.

The GRABBAG association (siblings) are essentially “one to many” relationships between person objects.

The system also creates reverse relationships for all of these automatically called BackReferences that provides an extremely powerful means of “walking” your datagraph.

References

The first example shows how to retrieve an object using a REFERENCE. We first get the Person with firstName “Bart” and then get the Person referenced by the object member “father”.

var bart, homer;
ff.getObjFromUri("/Persons/(firstName eq 'Bart')/father", function(response) {
homer = response;
});
//or, if bart has already been loaded on the client
homer = bart.father;
You can see References working here.

Grab Bags

This next example shows how to retrieve a set of objects from a Grab Bag by reference. We want to get all of Bart’s aunts on his mother’s side. To do this, we will first get the Person with firstName “Bart”, then get Marge from the “mother” member Reference, then get siblings from the Grab Bag referenced by the object member “siblings”.

var grabbags;
ff.getArrayFromUri("/Persons/(firstName eq 'Bart')/mother/()/siblings/(gender eq 'Female')", function(response) {
grabbags = response;
});
You can see Grab Bags working here.

Back References

This next example shows how to retrieve objects using Back References. There are three tests here – the first gets all BackReferences to “homer”. The second gets all the Person objects from the Grab Back that reference homer by the “siblings” Reference member. The third gets all the Person objects that refer to homer by the “father” Reference member.

var homer, allBackRefs;
ff.grabBagGetAll(homer, "BackReferences", function(response) {
allBackRefs = response;
});
var homer, allSiblingBackRefs;
ff.grabBagGetAll(homer, "BackReferences.Persons.siblings", function(response) {
allSiblingBackRefs = response;
});
var homer, allFatherBackRefs;
ff.grabBagGetAll(homer, "BackReferences.Persons.father", function(response) {
allFatherBackRefs = response;
});
You can see BackReferences working here.

Queries

This last example shows how to retrieve objects using some more advanced queries. There are five examples in this section. The first example will get all the Person objects that have a “father” Reference member (Note that the returned values are deduplicated). The second is basically the same, but uses the “mother” Reference member. The third example shows the use of a logical OR to get all of Bart’s granparents. The fourth example is similar, but adds another logical OR to get all of Bart’s grandparents. The last shows a complex query to get all of Ling’s cousins (the team’s favorite query).

var fathers;
ff.getArrayFromUri("/Persons/()/father"), function(response) {
fathers = response;
});
var mothers;
ff.getArrayFromUri("/Persons/()/mother"), function(response) {
mothers = response;
});
var grandfathers;
ff.getArrayFromUri("/Persons/(firstName eq 'Bart')/father or mother/()/father"), function(response) {
grandfathers = response;
});
var grandparents;
ff.getArrayFromUri("/Persons/(firstName eq 'Bart')/father or mother/()/mother or father"), function(response) {
grandparents = response;
});
var cousins;
ff.getArrayFromUri("/Persons/(firstName eq 'Ling')/father or mother/()/siblings/()/BackReferences.Persons.mother or BackReferences.Persons.father"), function(response) {
cousins = response;
});
You can see Queries working here.

Get the Entire Datagraph

Lastly, you can actually retrieve the entire datagraph in a single query by specifying the “depth” of the leaf-level items returned. This test will fetch the entire datagraph.

var datagraph;
ff.getArrayFromUri("/Persons/()?depthRef=3&depthGb=3", function(response) {
datagraph = response;
});
You can see the entire Datagraph here.

Final note: the source code includes a couple of Server Extensions that are used to populate/unpopulate the data for this app. You may find these useful samples, although they are rather brute force and inelegant in their current form.

FYI – FatFractal now provides textual term search for NoServer apps!


As a developer, I want to be able to retrieve information from my backend using search technology so that I can create even more powerful queries and get super fast responses.

You can see working code for everything below (here).
You can access the source code for the sample application (here).

We are really excited to announce that we our latest release includes full textual term search for all your data that is stored on FatFractal NoServer applications. This opens up an entirely new set of options for your queries and they are super fast to boot!

How it works:

We have recently started to use elasticsearch – a superb product by the way, and a future post will outline why we switched to it – as one of the data stores for your app’s data. Basically, as before, everything that is stored is fully indexed for you automatically without any configuration of your application or schema required. In addition, our implementation does not impose any additional overhead on any interaction with your data (Create, Read Update or Delete), but it has vastly improved the response times for all queries and we have added full textual term search as well.

How to use it:

We have added two new operators that you can use in your queries, “contains_all” and “contains_any”. These provide textual term search capability for queries that looks for, you guessed it, ANY submitted term matches or ALL submitted terms match.
For more information, you can find the documentation for the FatFractal Query Language (here).

Examples:

As usual, we have created a sample application to illustrate this, you can try the application (here), and the source code for the application is (here) and also play with the databrowser for the application’ backend (here).

The sample application uses a Movies collection that holds Movie objects. The movie objects have a member called “description” which we will be searching.

The model for Movie is as follows:

function Movie() {
this.title = null;
this.description = null;
this.year = null;
return this;
}
We have populated the collection with two Movies as shown below:

{
"title": "The Conjuring",
"description": "Paranormal investigators Ed and Lorraine Warren work to help a family terrorized by a dark presence in their farmhouse. Forced to confront a powerful entity, the Warrens find themselves caught in the most terrifying case of their lives.",
"year": 2013
},
{
"title": "Grown Ups 2",
"description": "After moving his family back to his hometown to be with his friends and their kids, Lenny finds out that between old bullies, new bullies, schizo bus drivers, drunk cops on skis, and 400 costumed party crashers sometimes crazy follows you.",
"year": 2013
}
So, now let’s create a couple of queries that use free text search.

Search using contains_any

First, we will search the Movies collection for any Movie objects whose description member contain any one of two terms (for example, “family” and “terrorized”). The query looks like this:

/Movies/(description contains_any 'family terrorized')
The code looks like this:
function searchAny() {
var eli = document.getElementById('movies-search-any-input');
ff.getArrayFromUri("/ff/resources/Movies/(description contains_any '" + eli.value + "')", function(movies) {
// handle success
}, function(code, msg) {
// handle error
});
}
Since “family” is contained in both “The Conjuring” as well as “Grown Ups 2” descriptions, this query will return both objects.

Search using contains_all

Next, let’s use the same search terms, but use the contains_all operator, which will search for any Movie objects whose description member contain both search terms.

/Movies/(description contains_all 'family terrorized')
The code looks like this:
function searchAll() {
var eli = document.getElementById('movies-search-all-input');
ff.getArrayFromUri("/ff/resources/Movies/(description contains_all '" + eli.value + "')", function(movies) {
// handle success
}, function(code, msg) {
// handle error
});
}
In this case, the query will return only “The Conjuring” as it’s descrption contains both terms and “Grown Ups 2” does not.

As mentioned above, these operations are extremely fast and efficient as any data that is stored on your backend is fully indexed automatically as soon as it is created or modified.

Have fun!

Kevin

FYI – Retrieve an object graph in a single query with FatFractal


As a developer, I want an easy to navigate data model (a datagraph) so that I can get the data I want with fewer API calls which increases my applications performance and saves me money.

We have added a new capability that allows you to retrieve an object graph (the object as well as all of its REFERENCES and GRABBAGS) in a single API call. This can improve the the performance of your application and reduce the “chattiness” of using your API.

Previously, you could retrieve an object from a URI such as:

http://telluride.fatfractal.com/tff39/ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4

For which you would get a response like this…[toggle title="Click to see response"]

{
"result":
[
{
"ffUserCanEdit": false,
"ffUrl": "/ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4",
"qa": false,
"ffRL": "/Showings",
"updatedBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"version": 1,
"updatedAt": 1346642155713,
"guid": "e-06R5_d_FfHXKHSCx-ky4",
"ffRefs":
[
{
"name": "theater",
"type": "FFO",
"url": "/ff/resources/Theaters/h3uX2Gt1j74w4yYbcifNq6"
},
{
"name": "film",
"type": "FFO",
"url": "/ff/resources/Films/KG09l8GvwRc9pUzXGMTEJ4"
}
],
"createdBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"createdAt": 1346642155713,
"datetime": 1315094400000,
"clazz": "Showing"
}
],
"statusMessage":"Retrieved 1 resources from /ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4"
}
[/toggle] The response includes the object itself (in ‘result’), and includes the URIs for any references (in this case, there are two). The FatFractal SDKs would then retrieve the referenced objects (requires a minimum of two additional queries, plus even more if these objects contain references or grabbag items themselves). That makes for a lot of round trips for the queries which can be expensive and adversely impact your application performance.

Get More for Less

Now you can access the entire datagraph from the object instance at a specified depth by adding the depthRef (depth for references) and depthGb (depth for grabbags) parameters:

http://telluride.fatfractal.com/tff39/ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4?depthRef=1&depthGb=1

This will return the object AND all of the object’s REFERENCES and GRABBAGS that the user has permission to read in a single response.

The response will now look like this…[toggle title="Click to see response"]

    "result":
[
{
"ffUserCanEdit": false,
"ffUrl": "/ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4",
"qa": false,
"ffRL": "/Showings",
"updatedBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"version": 1,
"updatedAt": 1346642155713,
"guid": "e-06R5_d_FfHXKHSCx-ky4",
"ffRefs":
[
{
"name": "theater",
"type": "FFO",
"url": "/ff/resources/Theaters/h3uX2Gt1j74w4yYbcifNq6"
},
{
"name": "film",
"type": "FFO",
"url": "/ff/resources/Films/KG09l8GvwRc9pUzXGMTEJ4"
}
],
"createdBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"createdAt": 1346642155713,
"datetime": 1315094400000,
"clazz": "Showing"
}
],
"references":
[
{
"ffUserCanEdit": false,
"filmType": 0,
"ffUrl": "/ff/resources/Films/KG09l8GvwRc9pUzXGMTEJ4",
"director": "Ben Affleck",
"ffRL": "/Films",
"country": "U.S.",
"version": 1,
"updatedBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"guid": "KG09l8GvwRc9pUzXGMTEJ4",
"updatedAt": 1346472626252,
"filmDescription": "While 50 Americans are held hostage by the Ayatollah Khomeini’s Revolutionary Guard, six U.S. Embassy employees hiding at the Canadian ambassador’s residence in Teheran stage a daring escape. “Extraction” expert Tony Mendez (Ben Affleck) enlists a veteran Hollywood make-up expert and producer (John Goodman and Alan Arkin, each giving disciplined comic performances) to create a sham sci-fi movie epic in the hopes of finding a way out of the country. Affleck directs with formidable precision and documentary-style immediacy, intercutting between turf battles between the CIA and Washington diplomats, the chaotic street violence during Iran’s revolution and the black comedy of spies pretending to be Hollywood types. This is thrilling, nail-biting entertainment.nIn person: Ben Affleck",
"duration": 120,
"title": " SNEAK: Argo",
"ffRefs": [],
"createdBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"createdAt": 1346472626252,
"madePossibleBy": "",
"year": 2012,
"actors": "",
"clazz": "Film"
},
{
"ffUserCanEdit": false,
"ffUrl": "/ff/resources/Theaters/h3uX2Gt1j74w4yYbcifNq6",
"ffRL": "/Theaters",
"version": 1,
"updatedBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"guid": "h3uX2Gt1j74w4yYbcifNq6",
"updatedAt": 1346359016787,
"createdBy": "Os0t9Mjm7FhPemzN0uQ8U7",
"address": "",
"createdAt": 1346359016787,
"name": "Chuck Jones Cinema",
"capacity": 500,
"longitude": -107.8478,
"latitude": 37.9361,
"sponsorString": "",
"shortName": "Chuck Jones",
"clazz": "Theater"
}
],
"statusMessage":"Retrieved 1 resources from /ff/resources/Showings/e-06R5_d_FfHXKHSCx-ky4"
}
[/toggle]

As you can see, the response includes the object itself (in ‘result’), but additionally, all objects referenced or contained in grabbags have been retrieved as well.

In your client code, using the methods provided in the various SDKs, the REFERENCE and GRABBAG items are automatically incorporated into your client object model, but now these are populated with a single request.

For more info on references, see the docs here.

Have fun!

Kevin

FYI – Queries that will make you smile! FatFractal’s natural language queries are very cool.


As a developer, I really want to be able to get the data back from my API in a single request so that my application performs better and I don’t get socked with excessive API requests that result from “chatty” APIs.

One of the things that developers love about FatFractal is the way that you can construct queries that can get you to what you want in a single round-trip.

First, the basics – FatFractal queries…

  1. …are written in an easy to create, easy to read, natural language style.
  2. …can be chained together to filter the results to only the results you want in a single request.
  3. …are able to “walk” your object graph in both directions.
  4. …provide the best of relational capabilities without the complexity.
  5. …are constructed as strings, not proprietary query objects.
  6. …includes a very useful random operator that for some reason, nobody else seems to offer.

Let’s say that you have movie Star, Movie and Theater objects and their respective collections (Stars, Movies and Theaters) and you want to get all the theaters that are showing movies that have a particular star acting in them within a defined distance from the user. The following explains how you can create a query string that can return that in a single request/ response to illustrate the power of nesting queries in a single construct:

/Stars/(firstName matches ‘(%3Fi)milla’ and lastName matches ‘(%3Fi)jovovich’)/BackReferences.Movies.stars/()/BackReferences.Theaters.movies/(distance (location, [57.5833, 3.8667]) lte 5000)?sort=rating desc, name asc&count=5&first=0

Creating or reading FatFractal queries goes left to right, with query constituents enclosed in parentheses with “traversals” and queries separated by forward slashes. So, breaking down the query above into pieces:

    /Stars/(firstName matches ‘(%3Fi)milla’ and lastName matches ‘(%3Fi)jovovich’)
    The first part (a query) will get the movie star(s) (Star object) from the Stars Collection with the firstName member ‘Milla’ and the lastName member ‘Jovovich’, using regular expressions that compare while ignoring case (requires proper encoding of the Java regex mode modifier ?i for ignore case to be added as %3Fi) and a boolean operator (and) to ensure a complete match. Note: we now have a set of Star objects, probably with a single object.
    /BackReferences.Movies.stars
    The second part (a traversal) will then get all the Movie objects that she starred in based on the BackReferences from the Movie objects in the Movies Collection to the star in the Stars Collection. Note: we now have a set of Movie objects.
    /()
    The third part (a query) is an empty query operator that says that will use the entire set returned by the traversal.
    /BackReferences.Theaters.movies
    The fourth part (a traversal) will then get the Theater objects from the Theaters Collection that are showing movies that Milla Jovovich starred in from the BackReferences from the Theater objects in Theaters Collection to the above set of Movie objects in the Movies Collection. Note: we now have a set of Theater objects, which is what we want.
    /(distance (location, [57.5833, 3.8667]) lte 5000)
    The fifth part (a query) will then filter that set of Theater objects by geolocation that are within 5 kilometers of the user.
    ?sort=rating desc, name asc
    The sixth part (a sort) will then sort them first by the rating member of the Theater objects in descending order, second by name member of the Theater objects in ascending order.
    &count=5&first=0
    Lastly, we then limit the result set of Theater objects to the first 5 items or less.

Summary:

So, basically we start from a Star object in the Stars Collection, walk the datagraph to Movies using a BackReference, then walk the datagraph again to Theaters using another BackReference, and then filter the set of Theater objects further based on location, order the results by both the rating and name member values and limit the number of responses to a specified maximum count. Voila – you get what you want, ordered as you want, all in one go with a single, easy to understand query – this simply cannot be done with any other BaaS provider.

Also, let me point out the random operator in the FatFractal query language as it is an example of a very common use case.

Let’s say you want to retrieve a random movie Star from the Stars collection. The query looks like this:

“/Stars/(guid eq random(guid))”

This is incredibly useful as otherwise, you would have to retrieve the entire set of Star objects from the Stars collection over the wire and then pick a random one from that set in your client code. Easy enough to program, but grossly inefficient in terms of client resources and network performance. Otherwise, you could *probably* retrieve a random item with two queries – first, by querying the collection to get a count, and if the API has an ordered ID (as many do), then you could generate a random ID in your client code and fetch that one (hoping that it has not been deleted). At any rate, getting a random object from a collection is a very common use case and I do find it bizarre that nobody else support this as easily as FatFractal does and forces you to use very inelegant client solutions.

One more point to make is that queries can be used in the context of GETS as well as within Server Extensions and Event Handlers.

All for now – have fun with your queries!

Kevin

For more information on FatFractal Query language, the docs are here.

For more information on how to write queries with FatFractal, the docs are here.

FYI – FatFractal supports many many to many many relationships (huh?)!


As a developer, I want to easily add many to many relationships between my objects without join tables so that I get a rich data model without getting data-model nightmares

With FatFractal, you can create amazing relationships between your objects. One of the things that makes this particularly useful is the ability to add multiple kinds of relationships from any particular object:

Example: If in your application.ffdl file that was created when you scaffolded your app, you include the following

CREATE OBJECTTYPE Comment (profile REFERENCE /Profile, comment STRING, relatedTo REFERENCE)
CREATE COLLECTION /Comment OBJECTTYPE Comment

This provides a ubiquitous “Comment” object that has two references and a “Comment” Collection typed to contain only “Comment” objects.

First, you will see a reference to the author public profile:

profile REFERENCE /Profile

This defines a relationship for the profile member which is bound to the /Profile Collection (which happens to contain only Profile objects thus “typing” the reference).

It also includes a reference called relatedTo:

relatedTo REFERENCE

This relationship is unique in that there is no Collection specifed, and thus no Objecttype implied. This means that the relationship for the relatedTo member can be to any object in any Collection of any Objecttype – i.e. is not “typed” and not bound to any particular Collection.

So now I have a comment object that has a reference to any other object in any other collection – pretty nice! All comments can are accessible via a single collection (/Comment) which is also nice.

But – what makes this truly useful is that you can also access comments via BackReferences to an object. For example, to see all comments (of Objecttype Comment) that relates to any particular object at all (as example, supR – a SupportRequest instance):

ff.getArrayFromUri(supR.ffUrl + "/BackReferences.Comment",
function(anArrayOfComments) {
// This returns an array of "Comments" that relateTo supR!!
},
function(onError) {}
}

Where the SupportRequest object model is defined as:

function SupportRequest() {
this.clazz = "SupportRequest";
this.subject = null;
this.body = null;
this.relatedTo = null;
this.profile = null;
return this;
}

Note that the SupportRequest does not even include anything about “Comment” in its definition. This means that you add relationships easily and intuitively without the pain of onerous data models – in short without the nightmares.

Hope this spurs some ideas for your application…

For more on the background of this, see Gary’s blog here.

For more details about FFDL, see the documentation here.

For more details about Queries, see the documentation here.

For more details about References and BackReferences, see the documentation here.

Kevin

Contact