Sign In
Register

Managing Data Persistence in the GameSparks Platform

Customers frequently ask how to persist various types of data on the GameSparks platform. This tutorial provides a guide to best-practices for managing data persistence on the platform.

Here's an overview of our recommendations:

By spending some time choosing the correct storage for your data to serve your specific requirements, you can improve the performance of your game and avoid scalability issues further down the line. Ultimately, this will lead to happier players who are more likely to return to your game again and again!

Player Data

The system Player collection offers two ways to store custom data (in addition to the standard data stored for a player such as currency, virtual goods, and so on). These are designed for storing key-value pairs, although note that the “value” can be a either a simple value, or a complex JSON object, allowing structured data to be stored if required.

scriptData

The first method for storing data in this way is to use scriptData. This can be set and retrieved using Cloud Code as follows:


Spark.getPlayer().setScriptData("myCustomData", { "subKey" : 1 } );

var myCustomData = Spark.getPlayer().getScriptData(“myCustomData”);

Any data stored in scriptData is available via the GameSparks API for authenticated players. That is, it will be returned to client devices in responses to requests including AccountDetailsRequest and ListGameFriendsRequest.

For example, sending an AccountDetailsRequest:


{  "@class": ".AccountDetailsRequest" }

Would return a response similar to:


{
  "@class": ".AccountDetailsResponse",
  "achievements": [
    "HS_ACH"
  ],
  "currencies": {
    "CURRENCY_1": 1100,
    "CURRENCY_4": 100,
    "CURRENCY_5": 100,
    "CURRENCY_2": 500,
    "CURRENCY_3": 100
  },
  "currency1": 0,
  "currency2": 0,
  "currency3": 0,
  "currency4": 0,
  "currency5": 0,
  "currency6": 0,
  "displayName": "Player One",
  "externalIds": {},
  "location": {
    "country": "GB",
    "latitide": 53.966705322265625,
    "city": "York",
    "longditute": -1.0832977294921875
  },
  "reservedCurrencies": {
    "CURRENCY_1": {},
    "CURRENCY_2": {},
    "CURRENCY_3": {},
    "CURRENCY_4": {},
    "CURRENCY_5": {}
  },
  "reservedCurrency1": {},
  "reservedCurrency2": {},
  "reservedCurrency3": {},
  "reservedCurrency4": {},
  "reservedCurrency5": {},
  "reservedCurrency6": {},
  "userId": "592837f2de3a8868577596f9",
  "virtualGoods": {}
}


privateData

privateData is similar to scriptData except this data is never sent to clients via normal API requests. This is always retrieved via Cloud Code only:


Spark.getPlayer().setPrivateData("myPrivateData", { "secretStuff" : 1 } );

var myPrivateData = Spark.getPlayer().getPrivateData(“myPrivateData”);

So, even if the player has some privateData set, only the scriptData will be returned using API calls – the result of an AccountDetailsRequest, for example, would be exactly the same as the example above even after setting privateData.

Using scriptData and privateData

These two mechanisms are perfect for when you have a small amount of data that is associated with a player:

However, storing large amounts of data this way is not recommended. Every time a player is accessed - one of the most common things to do on the platform - this data is retrieved and, in the case of scriptData, sent to clients in responses. This reduces performance and increases your bandwidth usage, leading to reduced responsiveness on the client. For this reason use these methods to store data only for data that is:


So what if you need to store larger volumes of data that is only accessed on-demand? In this case, you probably want to explore the following Custom Data section.

Custom Data

The Game Data Service was launched in January 2018 and how you store and manage custom data for your game is constrained by when you first created your game relative to that launch date:

Use Game Data Service! Even if you are working with a legacy game and although you can still work with Mongo Runtime custom collections, we strongly recommend that you make use of the new Game Data Service to store and manage custom data for your game. The Game Data Service has been designed with your game’s performance under high player loads very much in mind. In return for some straightforward set up work, where you define the indexes you'll use to query custom data, you can exploit a service that ensures any frequent data retrieval operations, such as rich-querying against your game’s custom data, is done optimally and efficiently. For more details, see the Game Data and Data Explorer pages.

Using the Game Data Service

Using the Game Data Service you can create custom Data Types to store large, complex, structured data, which can be accessed on-demand through Cloud Code:

Creating Data Types and Retrieving Data

The Data Type will be created when you create its first entry:

//Create entry and get its data object
var API = Spark.getGameDataService();

//Create entry, data is best accessed via ID
//Making the entryID as Spark.getPlayer().playerId is usually the best approach
var entry = API.createItem("dataTypeName", "entryID");
//Get the data object where custom data is stored
var data = entry.getData();

//Add new data to entry
data.exampleString = "foo";
data.exampleNumber = 234;

//Persist and return any errors
var status = entry.persistor().persist().error();

//If there are errors the entry would not persist and we can act on that information
if(status){
    //Output error script
    Spark.setScriptError("ERROR", status);
    //Stop execution of script
    Spark.exit();
}

You can later access that entry by referencing it by ID:

//Load API and get entry
var API = Spark.getGameDataService();

//Attempt to get entry
var entryObject = API.getItem("dataTypeName", "entryID");

//If error attempting to retrieve entry
if(entryObject.error()){
    Spark.setScriptError("ERROR", entryObject.error())
} else{
    //Get entry
    var entry = entryObject.document();
    //Access Data
    var data = entry.getData();
    var savedString = data.exampleString;
    var savedNumber = data.exampleNumber;
}

For entries that can't be returned through ID or need to return many entries that fit a condition, they can be queried. Note that you must first define the fields you want to use for querying a Data Type:

//Query Entry
//Load API and get entry
var API = Spark.getGameDataService();

//Example condition we wish to query
var condition = API.S("exampleString").eq("thisString");
//If we want to sort (yes)
var sort = API.sort("exampleString", true);
//Attemping to query
var query = API.queryItems("dataTypeName", condition, sort);

if(query.error()){
    //Output error script
    Spark.setScriptError("ERROR", query.error());
    //Stop execution of script
    Spark.exit();    
} else{
    //Create empty object
    var entryOBJ = {};
    //While there are still entries in the cursor retrieved from query
    while(query.cursor().hasNext()){
        //Get the entry
        entry = query.cursor().next();
        //Populate object with the entries. key = entry ID
        entryOBJ[entry.getId()] = entry.getData();

    }
    //Return entries via scriptData
    Spark.setScriptData("data", entryOBJ);
}

Best Practices and Guidelines

There are some important considerations to bear in mind when storing and persisting custom data using the Game Data Service:


Using Mongo Custom Collections

Custom collections allow you to store large, complex, structured data in MongoDB which can be accessed on-demand through Cloud Code.

There are two types of custom collections:

This section explains how to create Mongo custom collections, index those collections, and explains common errors and misconceptions:

Use Game Data Service! Even if you are working with a legacy game and although you can still work with Mongo Runtime custom collections, we strongly recommend that you make use of the new Game Data Service to store and manage custom data if you are continuing to develop your game further. See the previous Using the Game Data Service section

Note: To make sure your game performs and scales well, see GameSparks Best Practices.

Creating Mongo Custom Collections

Collections can be created on-the-fly in MongoDB. The first time you access a runtime collection, if it doesn’t exist, it will be created. For example:


var query = { "_id":"12345" };
var data = Spark.runtimeCollection("largeData").find(query);

This method will work, even if the “largeData” collection has never been explicitly created. However, it will return a cursor with no documents in it.

Storing data in a runtime collection is also achieved in Cloud Code:


var doc = { "gameState": { "gameType": "deathMatch" } };
Spark.runtimeCollection("largeData").save(doc);

Storing data in custom collections means that the data can be queried and (in the case of runtime collections) modified easily at run-time. It also provides all the power and flexibility that MongoDB has to offer.

Indexing Mongo Custom Collections

Custom collections can (and often should) be indexed for performance reasons. For example, if you always access the data by a field called “gameType” then you should index the collection as follows:


Spark.runtimeCollection("largeData").ensureIndex({"gameState.gameType":1});

This will create an ascending index on the “gameType” field. You should place the calls to ensureIndex for your collections in the Game Published system script, to ensure they are only called once for each collection (or more specifically, once per collection per game version that is published). There would be a slight overhead placing ensureIndex calls in a regularly executed Event script, for example.

Common Mistakes with Mongo Runtime Collections

There are two common mistakes to avoid when storing data in runtime collections:


If you do need to store binary data in GameSparks, such as uploading a file, you should probably be using Binary Assets.

Binary Assets

Binary assets fall into two categories:

Uploadables

Uploadables are files that, as the name suggests, are uploaded from a client device:

To upload data, a client device would first make an API call to GameSparks to retrieve an upload URL:


{ "@class": ".GetUploadUrlRequest" }

This will return a URL in the response:

{
 "@class": ".GetUploadUrlResponse",
 "url": "https://gsp-aeu001-se04.gamesparks.net/upload/351233XEAriw/56e91d8377588b04932481d8/51aada5dc51c4c7daf08a4b9a4136be5?gsstage=live"
}

The client device can then upload the binary data to the given URL, and receives an UploadCompleteMessage, which contains amongst other things, an uploadId:


{
 "@class": ".UploadCompleteMessage",
 "messageId": "5784cee777588b670617c090",
 "notification": true,
 "playerId": "56e91d8377588b04932481d8",
 "summary": "Your upload is complete",
 "uploadData": {
  "fileName": "51aada5dc51c4c7daf08a4b9a4136be5-portal.jpeg",
  "uploadId": "51aada5dc51c4c7daf08a4b9a4136be5",
  "fileSize": 9639,
  "origFileName": "portal.jpeg",
  "playerId": "56e91d8377588b04932481d8",
  "fileId": "ABC.12345"
 },
 "uploadId": "51aada5dc51c4c7daf08a4b9a4136be5"
}

You would then typically store this uploadId along with whatever other data you needed (for example, playerId, information about the level they were on, or any other relevant metadata) into your own custom runtime collection so you can retrieve it later.

Once uploaded, data can be retrieved (by either the same client or any other client connected to the game) by querying the custom runtime collection to find the uploadId, then sending a GetUploadedRequest:


{
 "@class": ".GetUploadedRequest",
 "uploadId": "51aada5dc51c4c7daf08a4b9a4136be5"
}

The response to this request contains a URL, which the client can then use to download the data:


{
 "@class": ".GetUploadedResponse",
 "size": 9639,
 "url": "https://gamesparksbinaries.blob.core.windows.net/upload-351233/51aada5dc51c4c7daf08a4b9a4136be5-portal.jpeg?sig=r4MGQ%2F9ulSftiHvDd08JWseo23s%2Bh8jftDWEaLxehVo%3D&st=2016-07-12T11%3A02%3A47Z&se=2016-07-12T11%3A17%3A47Z&sv=2015-04-05&sp=r&sr=b&gsstage=live"
}

Downloadables

Downloadables, on the other hand, are only available for downloading to client devices. These:


{
 "@class": ".GetDownloadableRequest",
 "shortCode": "DL1"
}

which returns the URL to download the asset from:


{
 "@class": ".GetDownloadableResponse",
 "lastModified": "2016-03-10T16:16Z",
 "shortCode": "DL1",
 "size": 300109,
 "url": "https://gamesparksbinaries.blob.core.windows.net/game-351233/1457626590618/axn-sean-bean-reddit-ama-6.jpg?sig=8gUTRizdXoWwKXVL5E28jG0uOXH5l7Dju%2FHT9HULsAg%3D&st=2016-07-12T11%3A06%3A04Z&se=2016-07-12T11%3A21%3A04Z&sv=2015-04-05&sp=r&sr=b&gsstage=live"
}


There is one final data store available to the GameSparks platform: Redis Data.

Redis Data

If you want a fast way to store simple data structures as key-value pairs, you also have access to a Redis instance for your game. Redis is:

Access to the Redis datastore is achieved, as usual, through Cloud Code. To store a simple set of values:


Spark.getRedis().sadd("MySet", 1);
Spark.getRedis().sadd("MySet", 2);
Spark.getRedis().sadd("MySet", 1);

This would result in two values (the numbers 1 and 2) being stored in a set against the key of “MySet”.

Redis is very powerful but can have a bit of a learning curve in comparison to other data stores. For a guide on what Redis is capable of and how to use it, the best source is probably the official Redis website at redis.io.

Privacy | Site Terms | Cookie preferences | © Game Sparks Technologies Ltd 2021
2022-01-06T18:45