6

I am working on an Android project and am currently trying to figure out how to deserialize some JSON from our APIs that includes reference cycles into an object graph, which I can then manipulate and store in a database. Let me give an example:

{
    "id": "24",
    "name": "Bob",
    "friends": [
        {
            "id": "13",
            "name": "Alice",
            "friends": [
                {
               "id": "24" // and we have a circular reference
                }
            ]
        }
    ]
}

Here, a person object called Bob is friends with person Alice, and Alice is in turn friends with Bob. Since the relationship is recursive, Alice’s friends relationship to Bob is not realized as a full person object anymore but only his id is provided.

What tools do you use to perform the above mentioned steps? I tried to implement the object mapping part with Jackson but failed to find a solution for the cycle requirement. I found an ongoing discussion about this topic that mentions JSOG which might be helpful, but our APIs are fixed and not JSOG compliant.

Basically what I am looking for is something like RestKit (iOS framework) for Android.

Maytham Fahmi
  • 31,138
  • 14
  • 118
  • 137
Rafał Sroka
  • 39,540
  • 23
  • 113
  • 143
  • I can build you a method to a parse the id, name, friends and return it as list or any data structure for further process, would that help? – Maytham Fahmi Jan 26 '16 at 20:01

3 Answers3

4

Once API is fixed, I'd implement it in this manner:

From DB perspective, I'd have 2 tables - UserTable and RelationsTable to keep all edges of your friends graph:
enter image description here

I.e. the idea is to keep Users in the one table and their relations in Relations table. It allows also to add some extra logic on top of it later (for example, user hides his connection or blocks someone, etc. - any possible edges of the graph). Also, it allows to mitigate issues with circular references.

As a framework to retrieve data from service & parse jsons, I'd use Retrofit.

First, I'd define UserBase and User classes:

public class UserBase {
    public string id;
}

public final class User extends UserBase {
    public string name;
    public List<UserBase> friends;
    // user's "real" friends, not just ids, fills from SQLite
    public List<User> userFriends;
}

where, as you can see, friends is a list of UserBase objects for Retrofit to parse the object from JSON and userFriends - the list, which we'll fill from SQLite manually in further steps.

Now, let's define some help-classes to operate with DBs:

public interface Dao<TItem> {
    void add(List<TItem> items);
    void removeAll();
    List<TItem> getAll();
}
....
public abstract class AbstractDao<TItem> implements Dao<TItem> {
    protected final SQLiteDatabase database;
    protected final SqlUtilities sqlUtilities;

    public AbstractDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        this.database = database;
        this.sqlUtilities = sqlUtilities;
    }
}

Now we need Dao's for RelatedTable and for UserTable:

public class UserRelation {
    public String mainUserId;
    public String relatedUserId;
}
...
public interface UserRelationDao extends Dao<UserRelation> {
    ...
    List<User> getFriends(String userId);
    ...
}
... 
public interface UserDao extends Dao<User> {
    ...
    void addWithIgnore(List<TItem> items);
    void update(List<TItem> items);
    void upsert(List<TItem> items);

    User getById(String userId);
    ...
}

Once it's done, we can actually implement this interfaces:

DefaultUserRelationDao class:

public class DefaultUserRelationDao extends AbstractDao<UserRelation> implements UserRelationDao {
    static final String MAIN_USER_COLUMN = "mainuser";
    static final String RELATED_USER_COLUMN = "relateduser";

    private static final String[] COLUMN_NAMES = new String[]{
            MAIN_USER_COLUMN,
            RELATED_USER_COLUMN,
    };

    private static final String[] COLUMN_TYPES = new String[]{
            "TEXT",
            "TEXT",
    };

    private static final String TABLE = "userrelation";
    static final String CREATE_TABLE = SqlUtilities.getCreateStatement(TABLE, COLUMN_NAMES, COLUMN_TYPES);
    static final String ALL_CONNECTED_USERS =
            "SELECT " + Joiner.on(",").join(DefaultUserDao.COLUMN_NAMES) +
                    " FROM " + UserTable.TABLE_NAME + "," + TABLE +
                    " WHERE " + RELATED_USER_COLUMN + "=" + DefaultUserDao.USER_ID_COLUMN;

    public DefaultUserRelationDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        super(database, sqlUtilities);
    }

    @Override
    public void add(List<UserRelation> userRelations) {
        try {
            database.beginTransaction();
            ContentValues contentValues = new ContentValues();

            for (UserRelation relation : userRelations) {
                sqlUtilities.setValuesForUsersRelation(contentValues, relation);
                database.insertOrThrow(TABLE, null, contentValues);
            }

            database.setTransactionSuccessful();
        } finally {
            database.endTransaction();
        }
    }

    @Override
    public List<User> getFriends(String userId) {
        Cursor cursor = database.rawQuery(ALL_CONNECTED_USERS, new String[]{userId});
        return sqlUtilities.getConnectedUsers(cursor);
    }
}

and DefaultUserDao class:

public final class DefaultUserDao extends AbstractUDao<User> implements UserDao {

    public static final String USER_ID_COLUMN = "userid";
    static final String USER_NAME_COLUMN = "username";

    public static final String[] COLUMN_NAMES = new String[]{
            USER_ID_COLUMN,
            USER_NAME_COLUMN,
    };

    private static final String TABLE = "users";
    private static final String SELECT_BY_ID =
            SqlUtilities.getSelectWhereStatement(TABLE, COLUMN_NAMES, new String[]{ USER_ID_COLUMN });

    static final String CREATE_TABLE = SqlUtilities.getCreateStatement(TABLE, COLUMN_NAMES, COLUMN_TYPES);

    public DefaultUserDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        super(database, sqlUtilities);
    }

    @Override
    public void add(List<User> users) {
        try {
            database.beginTransaction();
            ContentValues contentValues = new ContentValues();

            for (User user : users) {
                sqlUtilities.setValuesForUser(contentValues, user);
                database.insertOrThrow(UserTable.TABLE_NAME, null, contentValues);
            }

            database.setTransactionSuccessful();
        } finally {
            database.endTransaction();
        }
    }

    @Override
    public User getById(String userId) {
        return getUserBySingleColumn(SELECT_BY_ID, userId);
    }
    .....
    private User getUserBySingleColumn(String selectStatement, String value) {
        Cursor cursor = database.rawQuery(selectStatement, new String[]{value});
        List<User> users = sqlUtilities.getUsers(cursor);
        return (users.size() != 0) ? users.get(0) : null;
    }
}

To create our tables, we need to extend SQLiteOpenHelper and in onCreate() actually create tables:

public final class DatabaseHelper extends SQLiteOpenHelper {
    static final String DATABASE_NAME = "mysuper.db";
    public DatabaseHelper(Context context) {
        super(context, DATABASE_NAME, null, SCHEMA_VERSION);
    }

    @Override
    public void onCreate(SQLiteDatabase db) {
        db.execSQL(DefaultUserDao.CREATE_TABLE);
        db.execSQL(DefaultUserRelationDao.CREATE_TABLE);
    }
    ...
}

Now, I'd suggest to define LocalStorage interface with all possible actions with cache:

  • get all users
  • get user by id
  • add users
  • add connection between users
  • etc.

    public interface LocalStorage {
       User getUserById(String userId);
       void addUsers(List<User> users);
       ....
    }
    

and it's implementation:

public final class SqlLocalStorage implements LocalStorage {

    private UserDao userDao;
    private UserRelationDao userRelationDao;

    private SQLiteDatabase database;
    private final Object initializeLock = new Object();
    private volatile boolean isInitialized = false;
    private SqlUtilities sqlUtilities;

    // there database is
    //    SQLiteOpenHelper helper = new DatabaseHelper(context);
    //    database = helper.getWritableDatabase();
    public SqlLocalStorage(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        this.database = database;
        this.sqlUtilities = sqlUtilities;
    }

    @Override
    public User getUserById(String userId) {
        initialize();

        User user = userDao.getById(userId);
        if (user == null) {
            return null;
        }

        List<User> relatedUsers = userRelationDao.getFriends(userId);
        user.userFriends = relaterUsers;
        return user;
    }

    @Override
    public void addUsers(List<User> users) {
        initialize();

        for (User user : users) {
            for (UserBase friend : user) {
                UserRelation userRelation = new UserRelation();
                userRelation.mainUserId = user.id;
                userRelation.relatedUserId = friend.id;

                UserRelation userRelationMutual = new UserRelation();
                userRelationMutual.mainUserId = friend.id;
                userRelationMutual.relatedUserId = user.id;

                userRelationDao.add(userRelation);
                userRelationMutual.add(userRelation)
            }
        }

        userDao.addWithIgnore(users);
    }

    void initialize() {
        if (isInitialized) {
            return;
        }

        synchronized (initializeLock) {
            if (isInitialized) {
                return;
            }

            Log.d(LOG_TAG, "Opens database");
            userDao = new DefaultUserDao(database, sqlUtilities);
            userRelationDao = new DefaultUserRelationDao(database, sqlUtilities);
            isInitialized = true;
        }
    }
}

Last step - the actual usage of it:

//somewhere in non-UI thread
List<User> users = dataSource.getUsers();
localStorage.addUsers(users);
final User userBob = localStorage.getUserById("42");

NB! I'm heavily using here my custom class SqlUtilities. Unfortunately, it's way too big to post it here, but just an example to give some ideas what's inside - here's how getUsers(Cursor cursor) looks there:

.....
public List<User> getUsers(Cursor cursor) {
    ArrayList<User> users = new ArrayList<>();
    try {
        while (cursor.moveToNext()) {
            users.add(getUser(cursor));
        }
    } finally {
        cursor.close();
    }

    return users;
}

private User getUser(Cursor cursor) {
    User user = new User(cursor.getString(0));
    user.FullName = cursor.getString(1);
    ....
    return user; 
}
.....

I hope, you'll forgive me skipping some details (especially, regarding case, when DB has to be updated, when data is not full and besides getting it from cache, you have to retrieve it from server first, and then load it into the cache, etc). If any crucial part is missing - please, post it in comments and i'll be glad to update the post.

I hope, it will help you.

Konstantin Loginov
  • 15,802
  • 5
  • 58
  • 95
  • 1
    Thanks for the feedback, your ideas helped a lot already! However, your answer skips the parsing part of my question, how to handle cycles in JSON (using framework or custom code), that’s what I’m interested in the most. Any ideas? – Rafał Sroka Jan 26 '16 at 11:38
  • 2
    @Bearwithme actually, it's not :-) Retrofit parses your JSON by itself. Just keep properties of your `User` you want be parsed, as `public` and with same names, as in JSON, and the framework will do its job automatically – Konstantin Loginov Jan 26 '16 at 11:40
  • 2
    i.e. as you can see, I'm parsing Friend's list just as list of `UserBase` class objects (ids), and don't go deeper into the recursion – Konstantin Loginov Jan 26 '16 at 11:43
  • 1
    Ok, got it! Thanks again, the answer you provided is really good and helps a lot. I will wait until the end of the bounty with accepting. Thanks! – Rafał Sroka Jan 26 '16 at 14:19
  • 1
    @Bearwithme you're more than welcome :-) – Konstantin Loginov Jan 26 '16 at 14:20
  • Mate! they put this question on hold and cancelled the bounty. I am unable to award it to you :( So sorry! – Rafał Sroka Jan 28 '16 at 11:53
  • @Bearwithme That's all right! I hope, you'll solve your problem, good luck :-) – Konstantin Loginov Jan 28 '16 at 11:56
1

You can have a look into JSON-RPC. This is a good framework which supports JSON parsing and object mapping of complex object relationship.

Hari
  • 179
  • 6
0

I'd say you're trying to solve the wrong problem & the real problem is that your data representation is broken. As well as the circular refs problem its also inefficient in that each friend gets duplicated for each friendship. Better to flatten your list of people like this:

[
    {
        "id": "13",
        "name": "Alice",
        "friends": ["24"]
    },
    { 
        "id": "24",
        "name": "Bob",
        "friends": ["13"]
    }
]

Store the list in a HashMap<Integer, Person> (or SparseArray<Person>). Job done!

Reuben Scratton
  • 38,595
  • 9
  • 77
  • 86
  • Thanks for your feedback! However, I'm afraid you misunderstood my question. I hoped I was making it clear. As you can see our API proposal does not return full object indefinitely. When the object has already appeared in the response only its ID is returned. However, what I am asking for is suggestions for frameworks that are capable of parsing such thing, correctly establishing the relationships. – Rafał Sroka Jan 26 '16 at 11:01
  • Ok but your API is still the problem. You shouldn't have to even be thinking about this, just avoid circular refs in your model. – Reuben Scratton Jan 26 '16 at 11:40
  • 1
    I tend to disagree. Avoiding circular refs in the model by design is just one way of dealing with it. Good example is the JSOG standard in this case, there structure is not flat, circular references happen, and are described almost the same way I constructed my example JSON. – Rafał Sroka Jan 26 '16 at 11:43
  • Well we'll agree to disagree on the principle. Re. the practice, I hadn't heard of JSOG, is the Jackson plugin for it no good? https://github.com/jsog/jsog-jackson – Reuben Scratton Jan 26 '16 at 13:31
  • In the bottom of the question post, this link is also mentioned and concluded, as not working in this case. – Konstantin Loginov Jan 26 '16 at 13:35
  • Just modify the plugin to use a field named "id" instead of "@id"? Of course that will mean writing actual *code* which will clearly be a bit scary. – Reuben Scratton Jan 26 '16 at 15:07
  • @ReubenScratton would you mind posting it as another answer so me and the community can downvote/upvote? – Rafał Sroka Jan 26 '16 at 15:55