MARTIN GORNER: Hello, and welcome back
on Root Access Berlin.
I'm here today with Ignacio Coloma, the CTO of Koliseo.
IGNACIO COLOMA: Koliseo is a start up that aims to make
ticketing a pleasant experience,
as simple as possible.
MARTIN GORNER: So let us hear a demo.
And here on the Koliseo website, the nice thing is
that you have implemented sign ins with
all possible accounts.
I will sign in with my Google account.
And now, I want to create an event.
So the Factory will be opening its doors for the first time
in a couple of days.
And I want to invite people to the factory opening.
In the next steps, I can define the
details of my event.
I can also then create tickets.
And you have a platform for handling all the flow,
including registrations on the venue itself.
IGNACIO COLOMA: That's it.
You can do can control how the tickets are sold.
You can set the prices.
And you can also control the access when the day comes.
You can go and be in charge of the event.
And with your Android device, you can just
check out the tickets.
MARTIN GORNER: All right.
So now tell me, Ignacio, why did you
choose first App Engine?
And then also, you chose to use a NoSQL database, the App
Engine Data Store, even if you admitted that you had mostly
SQL background before?
IGNACIO COLOMA: That's right.
We come from a company with a lot of SQL experience.
But for us, when we were developing this project, we
wanted something that could scale, and that had--
we wanted all the uptime, but not the headaches.
We didn't want to be answering calls from our
customers all day long.
So we wanted to try something new.
MARTIN GORNER: And so yes, this is something I hear a lot
from start-ups.
That they go with NoSQL because it's fashionable.
And here, I would like to hear your expert opinion and help
people decide rationally between SQL and NoSQL.
So maybe we can try to come up with a list of checks that you
can do against your project requirements
to help people decide.
So the first thing to consider is of course scalability.
Can you tell us more about that?
IGNACIO COLOMA: When you're talking about scalability in
the SQL world, the things are more already thought out.
And the thing is you're going to have,
let's say, one server.
Now, your server is going to be scaling.
And at some point, you will have to put
different servers as well.
And you have to decide if you want to do vertical sharding
or horizontal sharding.
You have to decide if you're going to go with master
replicates.
When you want to make your databases scale, you have to
read a lot.
You have to read about the last 20 years of SQL
databases, differentiate the best practices from things
that are more or less outdated or something.
With NoSQL, the thing is you don't have this.
You have a data store that's going to work from the
beginning and it's going to scale for as many servers as
you like automatically.
Yes, you also have a learning curve.
But it's much shorter.
It's more focused on what matters.
MARTIN GORNER: So actually, the App Engine Data Store,
which is one example of a NoSQL database, what you say
is that it's multi-server from the start?
IGNACIO COLOMA: Yeah.
MARTIN GORNER: And it basically hides from you all
the multi-server complexity that is powering it?
IGNACIO COLOMA: High availability, load balance,
distribution of your data, everything is taken care of
automatically.
You don't have to think about that.
You don't have to configure that.
It just works.
MARTIN GORNER: The second point that I hear a lot from
people is that their feeling is that NoSQL represents their
data in a more natural way.
Was that the case also for Koliseo?
IGNACIO COLOMA: When you're talking about SQL databases,
you have standard solutions to standard problems.
So you're always watching your problems with the same lenses.
Let's say for example that you're talking
about the user account.
You have your user account.
And as you have seen, you can have multiple OAuth providers,
so multiple credentials.
How you do that with a SQL database?
MARTIN GORNER: So we tried to represent on this slide here,
and I think you need two tables, right?
IGNACIO COLOMA: Yes, right.
You need a master details table.
And what you're going to do is have a table with all your
users, and another one with all your OAuth IDs.
And you're going to relate one another.
With NoSQL, it doesn't have to be that way.
You have other options.
You have multi-valued fields.
With multi-valued fields, the same user can have multiple
values for the same field.
And you can handle that as if it was a single value.
So the same way you would have a user with a single OAuth ID,
for the same price, you can have multiple values.
And you don't need a separate table for that.
So for instance, here, my first user, Bill K., has two
accounts with which he can log in on to Koliseo.
And that means that any time I want to return a user that has
a given OAuth ID, in SQL, I have to do a join.
But also, whenever you want to retrieve
users by any criteria--
let's say I want all my senior users, like here, birth year
less than 1950--
then Bill K. will come up twice in the results.
Because the join actually created two Bill K's, one Bill
K. that has a Google provider, and one Bill K. that had a
Facebook provider.
So now, when I handle those users, I have to handle those
duplicates.
MARTIN GORNER: Yeah, that's a common problem.
And what I hear from people on a philosophical level is that
they like the fact that in a NoSQL database, a user entity
has its identity, it has its key, and you know exactly
where your user is.
IGNACIO COLOMA: Yeah, it's conceptually simpler because
you have a single entity with multiple values
for the OAuth IDs.
But you don't have to mix IDs just because of the data store
you're using.
In that sense, it's much simpler.
MARTIN GORNER: So a brief introduction about the code
snippets that we will be seeing during this episode.
When making Koliseo, you also made Simple DS, which is an
access layer for the App Engine database that allows
you, correct me if I'm wrong, that allows you to represent
your database entities using Java objects.
IGNACIO COLOMA: Yeah, that's right.
You can get all the mapping between Java and the data
store into this.
And you can match and mix between.
And you can use also a JSON
representation, or nested objects.
It takes care of all of that.
It's a little bit like Objectify.
You also have that in the Java world.
It came out more less at the same time as Simple DS.
MARTIN GORNER: So on scalability, NoSQL wins.
On the way of representing data, the philosophical
argument is that NoSQL kind of wins, although you
can argue both ways.
And now, the engineers designing NoSQL databases also
had to sacrifice some things.
So let's talk about that.
And first of all, maybe what is really super nice about SQL
databases is that they have the SQL language.
Anything I write that is syntactically correct will
execute, maybe slowly, maybe using all of my memory.
But it will execute.
It will return results.
IGNACIO COLOMA: Yeah, I missed that a lot.
I missed that a lot.
You don't have that in NoSQL.
Some of the queries that you can imagine won't execute
because you're using incompatible ways of querying
the data store.
For example, you cannot query a "different than." You cannot
query values different than two, for example.
MARTIN GORNER: Even before that, even if you want to
query by inequality, value equals something, you need an
index on that value.
IGNACIO COLOMA: Yeah.
If you're going to launch a query, you need an index for
everything that is inside of that query.
For example, if you are querying for a single value,
then you need an index for that value.
If you need two values in the same query, then you need a
composite index for these two values.
Also, if you're sorting, sorting by a value, querying
by another value, you need a composite index for that, too.
And that's sometimes a problem because every time you're
inserting, you're inserting the values, and also each
index that you're adding.
So sometimes, it's more the effort of writing the indexes
than it is writing actual data.
MARTIN GORNER: Yes, and also, based on the queries that you
want to run, you have to organize your data in specific
ways when you use a NoSQL database.
IGNACIO COLOMA: Yeah, sometimes you have to plan in
advance, how am I going to query this data before
you can design it.
So how am I going to organize my entities?
Which indices am I going to use?
Sometimes, you have to reorganize your data to be
able to execute this other way of querying this information,
this feature or whatever.
You need to do that.
But that's also common with SQL databases.
At some point, they're also refactored.
So it's no news for any of us.
MARTIN GORNER: But that is where database schemas come in
in SQL databases.
When I want to add a new field, I do one command, which
is alter table, and I have a new column in my table.
If you want to do this in a NoSQL database, it's slightly
more complicated.
IGNACIO COLOMA: Yeah.
If you want to modify your existing data in NoSQL
databases, you have to iterate over all the existing records,
adding new fields, removing existing fields, whatever, to
get what you would do with a single command with SQL
databases that would be alter table, and it's
automatically there.
MARTIN GORNER: And also, if I have a large project based on
SQL, usually I have a database administrator who is the
guardian of my schema.
And he locks it down to prevent random rogue engineers
from adding random rogue fields wherever they see that
they would like to have them.
IGNACIO COLOMA: Yeah, you won't have that in NoSQL.
For example, you wouldn't have a referential integrity.
So I can modify data by hand and leave the data store in an
inconsistent state.
That can happen.
You are not forced to have the same structure in all your
entities at all.
And sometimes, that's good because, for example, you have
data that depends on the kind that it has.
I have one field, or I don't have this field.
I have a different field.
You can use that to your advantage.
Or you can just use, in our case, in Java, you have your
persistent classes.
And that defines the properties that you have.
You can use it, or not, as you like.
MARTIN GORNER: So that would be another win for SQL, the
presence of database schemas.
And the last point that I would like us
to discuss is coherence.
NoSQL being multi-server by default, that is probably
where the trade-offs are the strongest.
Let's distinguish two different types of coherency,
first the coherency of transactions.
On a transaction, you want to be asset.
And you always want to be asset.
There is no trade-off there.
But to run a transaction on a NoSQL database, you have to do
something special.
IGNACIO COLOMA: Yeah, concretely in the App Engine
Data Store, you have entity groups.
Entity groups are based on that parent child
relationship model.
So you define your parent.
And everything that is handing from the parent, all of that
shares the same transaction.
MARTIN GORNER: And so you have to put all your entities into
one entity group before you can run a transaction on that
entity group?
IGNACIO COLOMA: It's more that you make clusters.
All the entities are a cluster are going to be part of that
transaction.
That was originally the only way you had in App Engine to
do transactions.
Actually, now it's more flexible, and now you can make
cross-group transactions, XE transactions that can put up
to five different entity groups in the same
transaction.
MARTIN GORNER: And also, the fact that you put entities
into one transaction group, that severely limits the rate
at which you can write to that transaction group, right?
IGNACIO COLOMA: Yup.
You have a limited rate of, I think, it's something like one
to five writes per second.
MARTIN GORNER: So that's very long.
IGNACIO COLOMA: Yeah, actually.
And you can create a great [INAUDIBLE]
there.
If you have a lot of users trying to push their
transactions in the same transaction group-- if it's
different transaction groups, it's fine.
But if it's in the same transaction group, you have a
limitation there.
MARTIN GORNER: Briefly, I would like to mention a second
type of coherency, which is the read your own writes
coherency, which in the SQL world,
again, is usually exact.
And here in the NoSQL world, it can happen that
you send the right.
And even reads that come after it still see the old value
until a certain point in time where the new value will
become visible.
So it's called eventual consistency.
IGNACIO COLOMA: Yes, that's right.
MARTIN GORNER: Was that a problem for you?
IGNACIO COLOMA: Not usually.
It's not usually a problem because we are talking about
one second of time between I do the change, and all the
queries in the world that are going to see that change.
There's always that small gap.
Maybe your listing, your user listing, is going to be one
second outdated.
That's not a big deal.
MARTIN GORNER: So it's something you
have to know about.
But in your opinion, that was actually a good thing, a good
constraint to relax?
IGNACIO COLOMA: Yeah, yeah.
MARTIN GORNER: Let's recap here the five points.
On scalability, NoSQL wins.
I think we all agree on that.
On the way you can structure your data--
IGNACIO COLOMA: Multiple valued fields
is a big win here.
And I really don't miss all those things.
But yes, we were talking before, maybe that's a problem
for some heavy users.
MARTIN GORNER: Yes, but on the other hand, SQL is a great
language if you have a DB admin
enforcing strong schemas.
That is something you will miss in the NoSQL world.
And the way you run transactions in NoSQL involves
slightly more work than in the SQL world.
IGNACIO COLOMA: Yeah.
MARTIN GORNER: All right, thank you very much, Ignacio.
And we will be seeing you again in the next episode.
In the next episode, we'll have tips and tricks to help
you overcome some of those NoSQL limitations so that you
can take advantage of the power of NoSQL and still work
around the limitations.
IGNACIO COLOMA: Yup, that will be fun.
MARTIN GORNER: So we'll see you next time.
Thank you very much, Ignacio, for being with us.
IGNACIO COLOMA: Thanks.
MARTIN GORNER: Thank you for watching Root Access Berlin.
Goodbye.
IGNACIO COLOMA: Bye.