Chapter 7 transcripts.

transcripts/ch7-mongoengine/1.txt

@@ -0,0 +1,77 @@
+00:01 Now we've got to a serious place in the course
+00:03 where we're going to write some real code
+00:05 and work with some realistic complex demos,
+00:07 and we're going to do that using what's called an odm, an object document mapper.
+00:11 So an object document mapper is like an orm, an object relational mapper
+00:16 but in NoSQL, we don't have relations we have documents
+00:19 so we're going to map documents to objects
+00:22 rather than navigate and traverse these relationships,
+00:24 and the one we're going to focus on for this course,
+00:26 my favorite one, I think is one of the best, if not the best
+00:30 is something called MongoEngine.
+00:33 So before we get to MongoEngine, let's just look at the overall goal here,
+00:37 and the features of the odm vary whether you're talking MongoEngine
+00:40 or something different, but they generally follow the same principles.
+00:44 So with PyMongo, this is the low level api
+00:47 we have our app and we have PyMongo, we'll talk to the database;
+00:50 so when we write a query that talks to PyMongo,
+00:53 we work in the Python api and we send Python dictionaries to it,
+00:58 which either have the prototypical documents in it
+01:00 or the high level operators, in place update operators
+01:03 and things like that like $addToSet,
+01:06 but in order to do that, we basically write in the raw MongoDB api,
+01:11 as we've seen the only real change that we go through
+01:14 to go from the raw shell api of Javascript over to Python is
+01:18 we're working with dictionaries and not json,
+01:20 and we're working with pythonic names,
+01:22 so insert_one has the underscore and lower case in Python, not in Javascript,
+01:27 but this means you're working at a super low level
+01:30 and for certain operations and some of the time this makes tons of sense,
+01:34 it is one of the faster ways to work with MongoDB from Python.
+01:38 However, most of the time, we much, much prefer having high level concepts
+01:43 in our application that enforce additional things
+01:47 that automatically use the best features of the PyMongo and MongoDB api
+01:52 without us having to think about how that happens.
+01:55 So that's when we can bring in an odm,
+01:57 we have the same thing, we got our app,
+01:59 now we're going to have our odm plus PyMongo,
+02:02 we're going to issue a query, but this time we're not going to write in raw api code
+02:06 we're going to actually issue the queries in terms of classes,
+02:09 think SQLAlchemy, think Django orm type of queries here.
+02:15 So we might have a book class given our previous example,
+02:18 so we'd go to the book and we'd say find based on the isbn equals this and so on,
+02:23 all right, so it's very similar to the Django orm
+02:27 and some of the other orms that you might be familiar with.
+02:29 So we work in these high level classes, and that's great
+02:32 and it translates of course down to the PyMongo api,
+02:35 what's better though, what's really great is
+02:38 it actually leverages the really advanced in place operator,
+02:41 so at least speaking of MongoEngine specifically now,
+02:44 if we say pull back a class, an instance of a class
+02:48 and we make a change to say for the book, we change the title and we call save,
+02:51 it's actually going to do a dollar set operation, in place update
+02:55 it's not just going to push the whole book back into the database
+02:58 with all the optimistic concurrency issues you might run into,
+03:01 no, it's going to make the changes in the best way possible.
+03:04 So we'll see that we'll be able to use these advanced operators
+03:08 without actually doing anything other
+03:10 than just working with classes and objects in memory,
+03:13 it's really really sweet; we'll also have additional features,
+03:15 it automatically works with indexes for us,
+03:18 it will automatically add type checking and other types of constraints
+03:22 that simply don't exist in the database, but can be managed at the class level
+03:26 in the object level and described there.
+03:28 So here's the string field, here's an integer field
+03:31 and the integer has to be greater than ten,
+03:33 all of that stuff can be done through MongoEngine, in our application
+03:36 but the concept of that doesn't even exist in MongoDb, right,
+03:39 so you get a lot more structured, a lot more safety in it
+03:42 by basically describing your schema in classes
+03:46 and long as you stick to one application or share that class definition
+03:50 across applications, you're in a much safer place
+03:53 than just randomly sending dictionaries at the database.
+03:57 So this odm style of programming, I find it to be extremely productive,
+04:01 very clear and quite safe, neat, fast for most of what we got to do,
+04:07 that's really my favorite way to work with MongoDB,
+04:09 and I hope you'll see the power of it
+04:11 and enjoy it after we go through in this chapter.

transcripts/ch7-mongoengine/10.txt

@@ -0,0 +1,119 @@
+00:00 Now it's time to service the car. So, we got a couple of options here,
+00:04 one possibility would just be let's get a service,
+00:08 like a random car we grab from the database,
+00:10 the other would be to go over here and implement list cars.
+00:13 So let's actually go ahead and do this list cars here,
+00:17 so we could hit list cars and then we can ask for the id
+00:20 or something to that effect of the car.
+00:23 Ok, so let's go over here and talk about listing the cars,
+00:26 we haven't done any queries yet, everything we've really done so far is inserts
+00:30 and that's really straightforward, right, we've seen that we create an object,
+00:33 we set some properties, they may be complicated properties
+00:36 but nonetheless, we just set the properties and we call save.
+00:39 For listing or any sort of query, we're going to come over here
+00:43 and we're going to do something different,
+00:46 so what we want to query are the cars, so we want to say car.objects,
+00:50 now there's a couple of things we can do,
+00:53 we can come over here and we can say filter and set something
+00:57 or if you're doing a single filter, you could actually do it right here,
+01:00 I'll do it the long way then we'll tighten it up,
+01:02 I'm not sure there's really any benefit other than whatever you feel like typing,
+01:06 do you want to be more explicit or do you want to have code be more concise.
+01:10 We could come over here and we could do a filter on the car,
+01:14 we could say I want to see only the cars
+01:17 that are a particular year or something like that,
+01:20 so I'm not sure we really need to filter it all, really all we want to do is sort them,
+01:26 so was say order by and then what we're going to do is
+01:30 we're going to say the thing that we want to order by,
+01:32 and for order by, we use string, so let's order by the year
+01:38 and let's just get with this for just a minute,
+01:41 so we'll say cars equal this, and then for car in cars
+01:45 we want to print out, let's say the make, the model and vin, something like this.
+01:53 So here we'll say car.make, car.model, car. vi_number,
+02:01 okay excellent, so that should print those out
+02:04 and let's do maybe a little extra line at the end.
+02:06 Let's go ahead and test this, and see if everything is working,
+02:09 so let's list the cars, excellent, surprise it's all Ferraries,
+02:12 remember, we are a Ferrari dealership, and this is not super helpful
+02:15 because it doesn't show us the year,
+02:18 I want to show you that the order is working, and if you don't see that
+02:21 you're not going to be able to verify whether this is working or not.
+02:26 So let's do this, we'll come over here and we'll say car.year,
+02:29 do it one more time, now we list, all right look at that,
+02:32 so 1991, 2000, 2005, 2005 and 2017, perfect it's working,
+02:36 but I'd kind of want to see the new ones first,
+02:40 although maybe in terms of service, seeing the ones is what we want,
+02:43 let's say we want to get them in reverse, so put a negative here,
+02:46
+02:50 here we go, now we're sorting the cars, newest to oldest,
+02:53 sorting by year, descending, okay, so that's working really well,
+02:57 so what we want to do is basically use this vin number,
+03:01 to go find the car we want to service,
+03:03 now that we can see the cars I'd like to go say I want to service a car
+03:06 and come over here hit s, and it is supposed to say okay
+03:09 what car do you want to service, I give it one of these,
+03:12 we're going to go to the database, find that car and then insert a service record to it.
+03:16 With that mind, let's go and think about this here,
+03:21 let's think about actually showing the service history of each car.
+03:25 Some number of service records like that,
+03:30 format what we want to print out is a len of car.service history
+03:34 now this is as far as we're concerned in memory just a Python list,
+03:37 so we don't need any database magics,
+03:40 like a straight up length will tell us what we're looking for.
+03:42 And then for each one of these, we can go and look at it
+03:45 so as in card.service history, this doesn't go back to the database a second time,
+03:49 that's part of the beauty of just pulling it back as part of the document
+03:53 so here we'll print, we'll do like an indent, something like this,
+03:56 it'll have let's say we're going to just show the name,
+03:59 let's do the price and the name, I think that might do it.
+04:04 All right, so we're going to s.price, s.description,
+04:09 I believe is what we called it, let's go and run this,
+04:13 we should see just zero histories everywhere,
+04:16 zero service records so not super interesting,
+04:19 but it should become interesting as we write this next thing,
+04:22 Alright, so now that we can find the cars,
+04:24 the next thing to do is actually service them,
+04:27 let's go ahead and write one more query here, and then I'll carry on,
+04:30 okay, so first thing we want to do, is we're going to get the vin,
+04:33 I'll say input, now in reality, the car would be sitting in front of us,
+04:43 here we don't really know, so we need to find a way to get the car right,
+04:48 so let's say this, we want to go to the car and we'll say car.objects
+04:52 now here's where we really actually are doing the filter stuff
+04:55 so we'll say filter— and this is pretty cool, we're going to go to this
+04:59 and we're going to basically pretend this function has named parameters
+05:04 that in the simple version represents the prototypical json objects,
+05:09 we saw from the api, from the Javascript and PyMongo api.
+05:14 So let's go to the car real quick, what is the thing we're looking for,
+05:17 vi number is what I called it, so we're going to say that equals to vin,
+05:22 if we wanted to also match the make = make, right
+05:26 model = model, whatever, if we had three things
+05:28 we want to do like an and on all three, this is how we would do it,
+05:32 but we don't, we just have the vin, now this will give us a list of cars that match this,
+05:36 in theory, this should be unique, we haven't got the indexes
+05:39 or any other way to make them unique yet, but we can get there,
+05:42 now we're expecting one back so we can come here and say first,
+05:46 and that should actually pull back the one car
+05:48 so instead of getting the cursor, we're going to get either a car or none.
+05:51 If we had none, that means it wasn't found,
+05:54 so let's say if not car, print car with vin not found,
+06:03
+06:07 okay so a little bit of error handling there, this doesn't throw an exception if it misses,
+06:11 it's just going to be none, all right, so then we'll say print
+06:14 we will service let's say car.make, let's see if this works,
+06:21 okay so list cars, let's go, let me put in service, say bad vin,
+06:29 car with bad vin not found, excellent
+06:31 let's try to service this one that has this vin that I copied,
+06:34 boom, we will service Ferrari excellent, excellent,
+06:37 so we'll go into the database and get it.
+06:40 Now that doesn't really prove very much, does it,
+06:43 that this is the model, there's only one Testarossa,
+06:50 so if I do this, it should say we're going to service the Testarossa, perfect,
+06:56 all right, so now, we have everything working beautifully in terms of our filter here,
+07:01 now I told you there's two ways to do this, if we have multiple clauses
+07:05 and you want to do multiple filters and multiple ifs and conditionals,
+07:08 just keep piling on the filters, but if you want a simpler version
+07:11 you can actually do it like this, as well, and let's just verify.
+07:14 Boom, see, we're going to still service that same car.
+07:19
+07:23 So now that we have it, how do we get this service record onto the car,
+07:27 you'll see there's actually two ways, and they're both pretty decent.

transcripts/ch7-mongoengine/11.txt

@@ -0,0 +1,83 @@
+00:01 Okay, so we can list the cars, we can now go and find the individual cars
+00:04 using this query and a little bit of error handling here.
+00:08 Now, let's go and actually add the service record
+00:10 so we are going to create a service = service record
+00:16 I think this is what we called it, no, service history, sorry,
+00:20
+00:24 we're going to import that, so we're going to allocate one of these
+00:27 and we just need to set a few properties, the price is
+00:29 let's just straight up input this— what is the price right,
+00:35 maybe we would look up at the type of service
+00:37 and it would have an associated price
+00:39 but this is just a demo, right,  just trying to get it to work;
+00:41 so this is the float, some kind of currency thing, I want to say service.date,
+00:48 we don't need to set the date, this should happen automatically,
+00:50 description is going to be input what type of service is this,
+00:56 and then the next one we want is going to be the customer rating,
+01:02 and this is an integer, how happy is our customer, I want to put a one to five,
+01:10 something like this to remind them that that's the scale they get to work with.
+01:14 Okay, we're just assuming this is parsable by float,
+01:17 this can be an integer right, we're not dealing with bad input.
+01:21 So that should work, let's just double check the service history
+01:24 these three we're setting, this one has a default, perfect.
+01:27 Ok, so finally all we have to do is we don't insert the service history directly
+01:32 because the service history is not a top level object
+01:36 it doesn't belong in its own collection,
+01:39 instead what we do is we go to the car and we say service history
+01:42 it's a list, so we're going to put it on it like this, like so,
+01:45 the ones we've changed the car we need to push that to the database like this,
+01:49 so that's all we got to do, we're just going to put the service history here and insert,
+01:54 let's go and run this and see what we get,
+01:57 alright, once again we're going to list the cars
+01:59 and notice we're going to work on the one that is really obvious
+02:01 this 2005 Testarossa, okay so let's service it,
+02:03 here's our vin number, excellent we're going to service it,
+02:06 what is the price of the service— this is expensive, 15 thousand,
+02:10 this is going to be a new engine, they blew the engine
+02:13 a customer was very happy, normally, new engine is 20 thousand
+02:17 and so we got him a good deal— bam, just like that, we've inserted it,
+02:21 let's try to list it I might still have that format wrong;
+02:24 no I got it sort of right, so we definitely got that to work there,
+02:29 let me just change that format like I said,
+02:35 there we go, actually let's just drop the currency, I'm sorry drop the decimal points,
+02:43 so here 15 thousand dollars, for our new engine,
+02:46 look at that, it's in there, let's go and actually do another service on that same car
+02:50 the price of this one is a 102 dollars, this is going to be a flat tire,
+02:55 and the customer was a little grumpy, felt like we took too long to fix the car
+03:00 but they still like us, they give us a three;
+03:03 so now if we list it you can see now there's two service records on the Testarossa,
+03:07 pretty cool, right, that's super easy, and we don't need to do a join to get this information,
+03:11 it comes built in even if we're doing horizontal scaling across ten shards it's all there,
+03:17 let's go look at Robomongo,
+03:21 it's a little small, what do you think, it does that sometimes, I don't know way,
+03:25 okay, here is our demo dealership, let's view the documents
+03:28 and we can say vi number, now of course we don't really need to do this,
+03:33 but we can, just to show you, we have tons of data,
+03:37 we look down here and now check this out, is that cool,
+03:39 so we've got our flat level things, here's the vin we just did a search by
+03:43 we have this embedded engine we already saw,
+03:45 we have our service history and moreover our service history is setting the date
+03:48 right, so right now it's like eight o'clock 04 seconds and 56 seconds right,
+03:53 the price of this is beautiful, just beautiful,
+03:57 so now if we do a query against this,
+03:59 we go and say show me the vehicle with this id number,
+04:02 we're going to automatically get the details about the engine,
+04:04 we're going to automatically get the details about their service history,
+04:07 without going back to the database or doing lazy loading,
+04:10 or joins or any of those kinds of things, because it's a sweet document database.
+04:15 Notice, also that some of these over here, some of these have a service history,
+04:22 some of them don't even have engine,
+04:24 this one doesn't have an engine or a service history,
+04:26 and that's probably ok, right, the schema is flexible in MongoDB,
+04:29 the schema is not flexible in our application, it's a class,
+04:33 it is what the shape of that class is, period,
+04:35 but in the sort of evolution of this as the data grows,
+04:39 you can think of this as like an older version of this record
+04:42 and here is a much newer one, it has an engine and a service history, right,
+04:46 but if we ask questions about show me the cars
+04:49 with the service history greater than this, you know, five or greater,
+04:52 these are not going to come up, they will just not appear in those results.
+04:56 So it's probably okay if we really need to upgrade them,
+04:59 to make sure the shape is exactly matching
+05:02 we could just run a script, the script would be a Javascript migrate script
+05:04 rather than a sequel dvl type migrate script, but a script nonetheless.

transcripts/ch7-mongoengine/12.txt

@@ -0,0 +1,104 @@
+00:01 So we are able to service these cars,
+00:03 however, there is something we might consider,
+00:05 it's probably fine for this type of application,
+00:08 but if there is contention around these documents,
+00:11 like multiple things that you're trying to update the same record
+00:14 you could run into some trouble here, right,
+00:16 we could have two separate threads, run this part,
+00:18 they could each enter some stuff here,
+00:22 there's actually because the input has super long delay here,
+00:24 and then we append it and save, and which every one got saved last
+00:27 would replace the other one and just throw that data away.
+00:31 We could add optimistic concurrency by manually implementing it
+00:34 and that would solve that problem,
+00:36 but we could actually make this perform better
+00:38 as well as avoid that problem entirely,
+00:41 so let's come over here and let's duplicate this, so hit command d in PyCharm,
+00:46 control d on the other Oss, so let's come down here and do this entirely different,
+00:53 so we're going to get rid of this part here,
+00:56
+00:59 and instead of saying this, since we're not pulling back the car,
+01:02 let's actually drop this bit here, so you can compare them directly,
+01:06 so we're going to ask for the vin, we're going to create the service history
+01:11 and now, instead of pulling the record back,
+01:14 making a change and pushing the entire document
+01:17 which could be like 200 K, we just want to move this data over,
+01:21 remember that in the raw api, we talked about the operators,
+01:24 we had $addToSet, and $push, so we want to use those operators
+01:35 and just say here, this service record put it onto the list
+01:37 that is on that document under service history;
+01:41 so that's what we're going to do now instead,
+01:44 so we're going to go car and I need to find an individual car,
+01:47 so I am going to do a query here, so I'll say vi number = vin, right,
+01:54 just like we did in our find up here, but instead of pulling it back
+01:57 we're going to take this and say update one,
+02:01 when we update one, I want to say something like say service history
+02:04
+02:09 and we'll say service, but how do I tell it to use the operator,
+02:12 this is the first time we've seen this but this,
+02:14 but this pattern you'll see recur over and over and over in MongoEngine,
+02:17 how do I tell it to use the operator?
+02:19 You'll see that there's a couple of times, a couple different situations
+02:22 where MongoEngine uses double underscores
+02:25 to like represent, not just the value but the value with an operator
+02:30 so the first one that are going to see is this push,
+02:33 we'll say push double underscore, and what that means is
+02:36 we're going to push service onto service history.
+02:39
+02:44 Up here before we had a way to say if you gave us the wrong vehicle id number
+02:48 we told you no, there's no car with this,
+02:51 there's no car with this bad vi vehicle id number;
+02:56 so what do we do here— well, this doesn't pull it back
+03:00 it just updates it if it finds it, so how is this going to work?
+03:04 We need to put our test over here and say if it wasn't updated
+03:08 so updated is if it comes back with one, but if it comes back with zero,
+03:11 that is a problem, the exact same problem
+03:14 we couldn't find the car with the vehicle id number;
+03:16 so now we don't need to append this and push the whole document back
+03:20 it's all in one shot atomically on the server.
+03:23 Beautiful, so this is a much higher performance and safer thing to do
+03:27 if we don't want any details about the car,
+03:29 we literally just know that this service goes on that service history for that thing,
+03:33 let's try again and see if it works.
+03:36 So first lease the cars, I'll grab the id of the Testarossa,
+03:40 and let's try to service it— the price is one two three,
+03:44 the type of service is waxing, so we decided to wash it for them,
+03:49 our customer thought the glean on that car
+03:51 was like nobody's business, very happy,
+03:53 there's the moment of truth, ready— boom, car with id not found,
+03:58 oh, it totally worked, it totally worked,
+04:04 I just have the wrong error message, wrong case here,
+04:06 let's do a listing and run it again do a list,
+04:12 now we have our waxing, what did I do wrong—
+04:15 there's a lot of ways we could check it,
+04:18 but if updated is exactly the one we don't want,
+04:20 I could say if not updated or if updated equal zero
+04:23 or if updated less than or equal to zero, things like this,
+04:26 let's just go with == zero
+04:30 alright, let's try it again, so we'll list our cars,
+04:32 notice I still have this, so we'll go on service it,
+04:36 I'll say I want to update this, the price is twelve dollars,
+04:39 this is going to be just the check up and the customer thought
+04:44 something was wrong, we couldn't find it, so they give us a three.
+04:47 So, see, did it work, notice there were no errors
+04:50 and now the checkup twelve dollars,
+04:53 also notice the order here is literally the order that we're putting onto that list,
+04:57 so very very good, we'll do one more, I guess I'll service something bad
+05:01
+05:06 car with bad then not found, all right now, sorry about that, we got this working.
+05:12 So we used the push underscore underscore
+05:15 now the other one, remember I told you about pythonic versus non pythonic
+05:19 you could use add to set, right if you want to do sort of a uniqueness thing
+05:25 the service history is because of the date field
+05:28 and I'm not even ensure about the embedded objects
+05:31 but certainly, the date field is always different,
+05:33 so there's no point in this, just do a push,
+05:35 but if you are pushing like tags or customer ids or something like that
+05:38 which could easily be determined if there is a duplicate
+05:41 then you could use that ad to set, but remember in Javascript
+05:44 it was add to set like this, here it's the pythonic variant as it probably should be.
+05:50 Okay, so depending on how you're working with your objects
+05:55 this might make sense or this might make sense,
+05:58 I guess I would tend to say prefer this one, this is safer and faster,
+06:03 so if you have no real reason to choose one or the other,
+06:05 why not safer why not faster.

transcripts/ch7-mongoengine/13.txt

@@ -0,0 +1,114 @@
+00:01 So we've really explored a lot of MongoEngine
+00:03 and we've built upon the foundation that we laid with the Javascript api
+00:07 and transferring that over to the PyMongo api;
+00:11 so hopefully, nothing you've seen has surprised you
+00:13 in terms of the types of queries that we're doing,
+00:16 it's just learning how MongoEngine surfaces that and turns it into objects
+00:20 was really what we were looking for;
+00:22 now, there are a few other things that we need to talk about
+00:25 that really we haven't touched on yet,
+00:28 the operators, we talked about the atomic update operators
+00:32 but not things like the greater than, less than, exists, doesn't exist,
+00:36 in set and so on, so we want to look at that;
+00:39 we also want to look at querying into subdocuments
+00:44 so if we go back to our MongoEngine here, we'll run this one more time,
+00:48 see, maybe we want to ask questions like
+00:51 show me the cars that have had some either really good service or really bad service,
+00:57 so we want to query all the way down into service history,
+00:59 into customer rating, and do a question like
+01:02 show me the ratings that were 5, show me those the ones that were 4,
+01:05 show me the ones that are less than 3, things like that,
+01:08 so how do we do this in this format that MongoEngine uses?
+01:11 So I've sketched out this little 'show poorly serviced cars'
+01:14 and it doesn't do anything, it just pulls back every car
+01:17 and prints it more or less like we had before,
+01:19 except for it shows the satisfaction in addition to the other stuff;
+01:22 so the question is how do I query it, let's just run it real quick,
+01:25 and I can say show me the poorly serviced cars,
+01:28 it doesn't matter what we put now, and it literally just lists all of them,
+01:31 and notice this one has a satisfaction of 3, 3, 5 and 3,
+01:34 so that we can do some queries, let's work on two other cars,
+01:38 let's work on the Ferrari 308 and this 2017 F40.
+01:42 So let's perform some service on this one
+01:46 and let's say this one got some amazing service, the price was 12 dollars,
+01:52 and we have a let's say monthly check up again here
+01:57 spelled right even, and they were just thrilled,
+02:01 so let's do our list really quick, and now, notice this one had a very happy one
+02:07 in fact, if I say the poorly serviced cars for a moment
+02:11 it's going to show that this one had a satisfaction of 5,
+02:15 okay let's suppose the 308 is not having such a good day, let's service it,
+02:19 and let's say that its price was 10 thousand dollars,
+02:23 the type of service was fender dent repair,
+02:27 so maybe the family went out of town and the teenage son stayed home,
+02:31 the son took the Ferrari out, found the keys and crashed it,
+02:35 so you can't blame the guy for being unhappy,
+02:38 but you know, what are we supposed to do, he came in unhappy,
+02:41 we tried to make him happy, but he was just not having it, so he had a 1,
+02:44 and now let's look really quick, just list everything still,
+02:48 so you can see over here, this Ferrari has no records,
+02:52 this one, this F40 2017, was very satisfied,
+02:56 the 308 very unsatisfied, and this Testarossa has some that are satisfied.
+03:01 Okay, so great, now we have the right variety of data,
+03:05 let's go over here and write the code that we were trying to write in the first place.
+03:09 What I want to do is I want to find the cars that had great service,
+03:12 so that's pretty easy to do, we saw that we could do like
+03:15 vi_number = 7, but what about, over here—
+03:20
+03:25 remember what we want to do, find the one with lots of them,
+03:29 we want to go into service history and down into service history,
+03:33 we want to find customer rating, how do we do that in this format?
+03:36 Well, it starts with this, service history, and what's the thing called down here,
+03:41 just do a copy to be sure it's identical,
+03:44 because you don't get an error if you get it wrong, just no results.
+03:46 So I told you that double underscore has special meaning
+03:48 we used it for the push operator earlier,
+03:51 we can also use it here to traverse the hierarchy,
+03:55 so service_history_ _customer_rating
+03:57 is going to go down and let's say this is going to match
+04:00 whatever level they passed in, all right, let's try this.
+04:03 So I want to find poorly service right now it assumes
+04:09 that we're going to enter a low number, but let's just run with it for a minute,
+04:12 let's say I want to find the ones with level 1,
+04:15 all right, so it was this Ferrari 308 here,
+04:18 and I think that's the only one that has level one,
+04:21 let's go and run the poor but ask for 5,
+04:24 so like I said, bad name, servers at a level or whatever;
+04:28 now we have two, right, we have this Ferrari F40 with this here,
+04:32 and we have the Testarossa, which some of the time
+04:36 at least had really good service, the person was super thrilled.
+04:39 So that's how we search into those subarrays, we used the double underscore,
+04:45 so double underscore we used it for push onto a thing,
+04:48 we use it to navigate a hierarchy,
+04:50 the last thing that we really are looking for is
+04:53 we would like to find the cars that say have
+04:55 below excellent service or something like that,
+04:58 so let's change this a little bit, max level of satisfaction are we looking for;
+05:07 so we could say 1 and that's a really bad one,
+05:09 if we could say 3, and we could intend that to be 1, 2 or 3, as the level, right,
+05:14 so it's not going to work this way now, it's just going to be straight up a quality.
+05:18 So, once again, how do we do it in the Javascript api or PyMongo—
+05:24 we would use something like this, we would say that,
+05:27 we would say service_history.customer_rating
+05:38 and then here instead of giving a number we would give it one of those operators,
+05:44 we would say $ lte (less than or equal to) : level, right
+05:49 so how do we do that here— well, we want to use this operator
+05:55 and we're going to do that again with the double underscore,
+05:58 so we'll say double underscore __lte,
+06:00 but here's the thing, the query operators go on the end,
+06:03 the update operators go on the beginning, remember push was like this
+06:07 so the order varies, for better or worse,
+06:10 I think it has to do with the fact that the operators here go to the right in the raw api,
+06:15 and the push one goes to the left, so it's kind of trying to mirror that.
+06:19 All right, let's run this again. So let's see the poorly serviced cars,
+06:25 let's try again for 1, we should see just the 308
+06:27 because that's the only one with that level, boom, there's the 308.
+06:31 Let's look for it again, I want to find all the cars with 3 or below,
+06:35 remember, if I scroll this up a little bit, we're doing lte less than or equal to 3,
+06:40 bam, look at that, we got the 308 and we got the Testarossa,
+06:45 which some of the time did have this, all right,
+06:47 if I put 5, we would just get all of them.
+06:50 So you can see that we can use the double underscore to traverse the hierarchy,
+06:53 we can use the double underscore for the operators,
+06:55 and in fact, we can use the double underscore for multiple meanings
+06:58 in the exact same thing, right here, traverse service_history.customer_rating
+07:04 and then apply the operator less than or equal to the value that we set.

transcripts/ch7-mongoengine/14.txt

@@ -0,0 +1,117 @@
+00:01 Let's look at one final thing, I think it is not beautiful
+00:03 but knowing about it and expecting it is really, really important,
+00:07 not in the beginning, but as you evolve your application,
+00:10 you'll end up with some funky complications.
+00:13 I actually chose the cars that I wanted to update very, very carefully,
+00:17 let me run this again, if I list the cars, notice here in particular state
+00:22 far away from this Ferrari F40  from 2005, there's only one of those, right,
+00:27 notice the id it's d15 and ends in 7e, if I try to list the cars again
+00:31 there's that one but oh, it ends in ae, list the cars again—
+00:36 now it's ending in a1, what is up with that?
+00:43 Just to be clear, the other ones are not changing,
+00:45 like 0f that's always the value, for the first one 0f,
+00:51 there's not a problem with Mongo or anything like that,
+00:54 what is going on here is this car was inserted into the database
+00:59 when we just had a little bit of our class to find here,
+01:02 remember in the beginning, we didn't have this default concept
+01:06 when I first introduced it, and somewhere along the way
+01:09 after we had inserted a few cars, then we added this,
+01:12 let's look at Robomongo.
+01:17 If you flip through here, you'll see almost all the cars have a vi number,
+01:20 vehicle id number, vehicle id number
+01:24 except for that F40 from 2005, right at the top— none.
+01:29 Because when it was inserted, there was no definition for a vi number
+01:34 what the heck was that thing anyway,
+01:36 how was it supposed to know that was not here yet but would eventually exist;
+01:39 so we can do a couple of things,
+01:41 the reason you keep seeing this changing numbers is that there's a default value
+01:45 and basically it gets created every time
+01:50 it comes back from the serialization layer,
+01:53 but it doesn't get set from the database
+01:55 because there is nothing to set it to,
+01:57 so every time it goes back, it reruns that lambda
+01:59 and gets a new value and we're not saving it.
+02:02 So basically what we need to do is we need to upgrade our documents,
+02:04 now sometimes like I said, this doesn't matter,
+02:07 but this one where we kind of counted on a default value to be there,
+02:10 and then it wasn't, well that's unfortunate.
+02:13 So here's something, there's a couple of things we could do,
+02:17 I could simply come up here and write the script in Javascript
+02:20 and apply that, that's one option,
+02:22 another option is let's go here and let's write make sure
+02:25 we just below configure Mongo I'll say update_doc_versions, or something like this
+02:32 define that function here, and what we can do is something like this
+02:37 it's not exactly in a work, but it'll give you the idea what we're after;
+02:41 so we'll say for car in car.objects so basically
+02:44 let's look through everything in this collection and let's save it back,
+02:48 I'm going to run this, and let's list the cars and see if we solved it,
+02:53 I really wish it wasn't at the top but there is, 19, 02
+03:02 what's going on, well, it turns out that unless we somehow forcibly change this object
+03:09 it's like hey, this object is not dirty and we don't need to push it in there.
+03:12
+03:16 And say mark as changed, vi number and let's try it again—
+03:20
+03:26 here we go, so we told it that it changed
+03:30 sadly even though it generates a default value
+03:32 it doesn't look back into the database which I guess would be super expensive,
+03:37 it just says hey someone changed this, right
+03:39 it didn't really trigger that, it came from the databases none and then we set it,
+03:43 so it doesn't know to push that through, so you have to do this little bit of a trick here
+03:46 to say mark has changed, and PyCharm says you're not supposed to do that,
+03:52 let's just tell it hey, don't make me look bad,
+03:55 we have to do this you understand it, right?
+03:57 Very well, okay, so now we've got this save back to the database,
+04:01 we only want to run this the one time, right, we don't want to run at all the time,
+04:05 this is just like a one time upgrade of our documents
+04:08 and if you have a 100 thousand records, probably fine,
+04:11 if you have a billion records this is not how you want to do it,
+04:14 you want to do some kind of in place updater or something better, write the script,
+04:18 so let's run this again, and now we should see our car here,
+04:23 this is the 2005 F40, you know what,
+04:26 it is time for new tires, let's service this puppy.
+04:30 Now we come down here and say I don't know how much new tires are in Ferrari,
+04:33 but let's say they're 2000, new tires,
+04:35 the customer is pretty happy they had the low profile ones they were looking for
+04:39 but they could have been like a certain kind, who knows, whatever.
+04:43 Perfect, that worked, now if we list our cars again,
+04:46 you can see that this one that was basically, we couldn't get to
+04:50 because its vin number kept changing is now fixed
+04:54 and that's because when we reran that we said hey,
+04:58 force of the default of the vin number in there,
+05:01 notice that none of these other ones changed,
+05:03 it did write them back to the database, but it wrote them back
+05:06 in exactly the way they were before, so nothing changed there,
+05:09 I'll just run it one more time, this second one is,
+05:12 I'll just copy this and we can go pull it back in a second,
+05:14 so if we put this back one more time, and then we try to service this car
+05:23 one dollar test service they were pretty thrilled with that
+05:30 here you can see the test service, okay.
+05:37 So the ids are not changing when we do this,
+05:39 it's just if they're not there they are created.
+05:42 Alright, so if that seemed kind of annoying, I'm sure it was annoying,
+05:46 but let's think how this would be in a relational database,
+05:50 what would have happened if this was a say SQLAlchemy or something to that effect,
+05:55 or if this was some other thing, we would have a lot more trouble
+05:58 evolving from one to the next, right,
+06:01 so we wouldn't have the problem of
+06:03 hey here's a car that doesn't have a vin number,
+06:05 because if we didn't actually go and manually changed the database
+06:10 every time we added something here, when we added this
+06:13 we would have had to go back to the database
+06:17 and do like a migration or data transformation SQL query
+06:20 to actually change and add this column, same thing for this,
+06:23 but none of that was required, it was just this one case
+06:26 where we went back in time that we had to do a little bit of work here.
+06:29 So, sometimes you still have these scripts you've got to run,
+06:32 sometimes you still have these changes, you got to do
+06:35 and consider the version history,
+06:37 but it's much much less often than with relational databases
+06:41 where every little change requires a script
+06:43 or it's just oops things are out of sync, bam we can no longer work,
+06:46 but I did want to point that out to you that look,
+06:49 you're going to have to be really careful, some of the time.
+06:52 as these things evolve, how are you going to deal with the fact
+06:54 that in the database there is this thing that has no vehicle id number.
+06:58 If we're using PyMongo, it would have just come up as none
+07:02 or key error or something like that, it would have been a little more obvious
+07:06 but that's just one of the trade-offs you get with these ODMs.

transcripts/ch7-mongoengine/15.txt

@@ -0,0 +1,44 @@
+00:00 One of the very first things if not the first thing
+00:03 that we need to do is register our connections.
+00:05 So it's really straightforward, we just import MongoEngine
+00:08 and then we call register connection,
+00:11 and you want to give this connection an alias or set it as the default,
+00:14 and then we're going to set the name equal to the name of the database.
+00:18 Here we're calling this one core, I like to be very explicit and say
+00:22 everywhere that you are working with a connection or a database really
+00:26 you name that explicitly in your code
+00:29 as we'll see later when we get to the classes.
+00:31 So we register connection, and we set the alias to core,
+00:35 and the db we're going to say name = dealership.
+00:38 Now, this worked well if you're just connecting on local host
+00:41 no authentication, default port all that, right
+00:44 we just let everything else fall through the cracks.
+00:46 When we get to the production deployment,
+00:49 well that's not really going to fly anymore, we're going to need ssl,
+00:52 we're going to need to enable authentication
+00:54 and pass credentials and all that kind of stuff,
+00:56 so we can use a more complicated variation here,
+01:00 where we do basically the same thing, but we create this dictionary
+01:03 that has the additional elements, now it doesn't have to be a separate dictionary
+01:07 you could just set them explicitly, but it turns out that sometimes
+01:10 if you want to like put this into your log or things like this, it's kind of handy,
+01:14 so we're going to basically set a username, password, host, port,
+01:17 authentication source is almost always admin, not always,
+01:21 it's either the database you're working with if it's a local user
+01:24 or if it's a server wide user you're using to be on admin
+01:27 authentication mechanism is scram-shah-1,
+01:30 or you can change it that's the default and ssl is true,
+01:33 and in this case, we might be using a self signed certificate
+01:37 which is totally good for the encryption, but it's going to fail
+01:40 and say we don't trust the certificate, if we trust the server
+01:43 you can go with ssl cert requires none,
+01:47 or if you want to make sure you have one, trust its certificate, omit the last line.
+01:50 And then we just use **data basically to pass those
+01:53 as keyword arguments to register connection
+01:56 and notice, each step I'm saying get the user from config or environment
+02:00 so this could be in a web app where these values are stored in the config file,
+02:04 you don't want to put them in source specifically
+02:07 you don't want them checked in anywhere ever,
+02:09 you could say get them from, you can put them in environment variables
+02:12 on your server and then grab them at runtime
+02:14 out of the environment and set them here.

transcripts/ch7-mongoengine/16.txt

@@ -0,0 +1,59 @@
+00:00 The way we primarily work with MongoEngine
+00:03 is we create classes and we map those to collections.
+00:06 So here we started out with a really simple car,
+00:08 we have a class called car and anything that maps to a collection
+00:13 is a top level document must be well derived from mongoengine.document
+00:18 and then we set up just all of the fields, these could be simple
+00:21 or as we saw they could be nested rich objects,
+00:24 all the ones listed here are simple, so we have string, string int, int and string.
+00:29 So we just do that mongoengine.stringField and so on.
+00:31 So this worked pretty well, but we said it would be nice
+00:34 if we could express that some of these are required,
+00:36 that some of these have default values and things like that,
+00:39 so we can come in here and we can say the model, the make, and the year
+00:43 these are all required, just say required = true you must type them in;
+00:47 mileage, we might be happy to go with zero for default
+00:50 this is new cars, things like that, so zero is a good default there,
+00:54 the vi number, the vin number is more interesting,
+00:57 we want to generate a large unique alpha numeric string
+01:01 automatically when a car is created,
+01:03 so we'll say default equals and will give it some kind of callable
+01:06 in this case a lambda that returns a string based on taking the uuid4,
+01:11 turn it to a string, drop the dashes, things like that.
+01:14 So this worked really well for generating our car
+01:17 and we didn't even have to set the vin number, that just got done automatically.
+01:21 Finally, we said look, our cars also are going to contain an engine
+01:24 and I don't want to go and do a separate query to a separate table
+01:28 or separate collection specifically,
+01:31 to find out details about the engine and store like the car id in the engine,
+01:34 so instead, we're just going to embed it straight into the car,
+01:38 you have a car, you have the entire details, precisely.
+01:40 So we did that by first creating an engine class
+01:43 and that engine class has to derive from mongoengine.EmbeddedDocument
+01:48 not document, don't make that mistake, EmbeddedDocument
+01:51 and then we're going to set the type of it here in the car
+01:53 to be an embedded document field,
+01:56 the embedded document feel takes two things,
+01:57 the type that you're going to put there so the engine class
+02:00 and whether it's required is optional, right,
+02:03 but we're going to say at least here yes the engine is required.
+02:06 We also wanted to store the service history,
+02:08 a set of rich documents modeled by service records,
+02:11 so again here's a class derive some embedded document
+02:13 but this time it's not one thing, it's a list of them,
+02:16 so we have an embedded document list field
+02:18 and this basically starts out as an empty list
+02:21 and then as we wish we can append these service records to it
+02:25 and then save them back.
+02:27 So if we have our car model like this and we put one into the database
+02:30 it's going to come out looking like this,
+02:32 we'll have an id, we'll have a model,
+02:34 bunch of other flat elements up there, flat fields
+02:36 we have our vin number generated as 9501, from that lambda expression,
+02:41 the engine has four properties horse power, liters, miles per gallon, serial number,
+02:45 and that is modeled by that engine object,
+02:48 and notice the curly braces, this is an embedded sub document here
+02:52 and the service history, notice square bracket this is a list or an array in Javascript
+02:57 and it has a bunch of sub documents that are the service history.
+03:00 So with our car modeled in Python on the left
+03:02 what we get here on the right is actually what you'll see in MongoDB.

transcripts/ch7-mongoengine/17.txt

@@ -0,0 +1,29 @@
+00:00 We saw that inserting in MognoEngine
+00:02 is super super straightforward, it's really delightful,
+00:05 so here we're going to create a car, but remember
+00:07 the car requires an engine, the engine is required
+00:10 and the engine must be an instance of an engine object.
+00:13 So we're first going to create an engine,
+00:15 set things like the horsepower, the liters, is the miles per gallon,
+00:19 so notice this is a chevy bolt which is an electric car
+00:22 so we just ramp the mile for gallon like super high,
+00:25 liters is zero because how many does an electric engine have- none,
+00:28 and I'll say it's a 120 horsepower, I really have no idea.
+00:32 Then we're going to create the car, its model is a bolt, its make is a chevy
+00:35 and the year is 2017, and then we just pass the engine along,
+00:38 right engine = engine, one is a keyword value and one is just the name of the variable.
+00:43 So then we have our car, and right now the id of the car is not stored in the database,
+00:47 so we hit save and boom, now we have like a car with its id and its default values set
+00:52 all of those things stored in the database.
+00:55 So this is great for inserting one car
+00:58 but if you are going to insert a thousand or a hundred thousand or a million cars
+01:03 let me tell you, this is the slowness right here, you do not want to do this;
+01:07 there's a much better way, maybe you don't take a million inserts at once
+01:10 maybe you bulk it up and do like 50 at a time or a 100 at a time,
+01:13 but if you are going to do some kind of bulk insert how do you do that?
+01:16 Also super easy, let's suppose we have a list of cars that we want to insert
+01:20 and I'm not showing how you initialize the cars, but same as above basically,
+01:24 but skip the save step, so we're going to get car one, car two,
+01:27 we want to insert a bunch of them we just go car.objects.insert and give it that list
+01:32 and boom it does a bulk insert in MongoDB,
+01:35 which if you're inserting many items is much much faster.

transcripts/ch7-mongoengine/18.txt

@@ -0,0 +1,63 @@
+00:00 We saw querying in MongoEngine was really quite straightforward
+00:04 and mostly was done by way of interacting with the class
+00:07 and the arguments, not breaking down and working
+00:12 in the actual MongoDB or even PyMongo api;
+00:15 so let's look at a couple of options,
+00:17 a couple of ways in which we might do some queries.
+00:19 So here we have a function called find_car_by_id
+00:21 and notice it's taking a car id which we're using type annotations or type ins
+00:25 to indicate that this is an object id that comes in
+00:28 and what we give back, what we're returning is an individual single car
+00:31 and really for the type ins to be entirely correct
+00:34 you should say optional car, because it may be none as we saw.
+00:37 So we are going to say car.objects and then filter(id = car_id)
+00:41 so what we're doing is saying we're looking for the car object
+00:44 that has id = car_id, now this is by primary key basically
+00:48 so you expect it's one or zero, we do a first
+00:51 so that actually returns either the object or none.
+00:54 One thing to know is in the database it's _id and
+00:57 in MongoEngine they are like forget the underscore it's just straight up id,
+01:01 so minor difference there,
+01:03 we also saw that if you have just one filter, like a really simple thing
+01:06 you could just say objects_id = card_id
+01:08 and don't have to do the filter step, but I kind of like this explicit style.
+01:11 So we're going to say we're looking for one or more fields
+01:16 we're only passing id but we could pass id and vin number and other types of things
+01:21 and we call first to get one or nothing at all,
+01:26 next, we might want to query by subdocuments
+01:29 or things contained in a list inside of that document, stuff like that;
+01:33 so we can also do this with MongoEngine.
+01:36 Same type of thing card.objects.filter, however what goes in here
+01:40 is no longer just the straight name,
+01:43 in fact, we're going to use the double underscore
+01:45 to traverse that hierarchy, so we're going to go down to service history,
+01:50 then within service history we're going to look at customer rating.
+01:54 Now, if we're going to return this, we maybe don't want to return it active cursor
+01:59 we may either want to use a generator
+02:02 or here what we're doing is we're actually creating a list
+02:04 we're saying list of cars, that way by the time this function is done executing
+02:09 it actually is going to entirely have finished whatever it's doing with the database
+02:14 and you'll basically be done with the database by the time you leave this function.
+02:19 Notice we're also using type ins to say this takes a list of cars,
+02:22 this time we didn't call first we're just converting all of the responses to that list.
+02:27 All right, finally this one that we just looked at
+02:31 was looking exactly at the customer rating of 4,
+02:34 but here we want to know like show me all the cars
+02:37 that were not rated with great service, right
+02:39 that is 3, 2, 1 or 0, if that's a possibility,
+02:43 and in fact, we're going to use not pull all the cars back
+02:47 but we want to know like as a percentage how are we doing,
+02:50 how many cars that we have that had sub amazing service, versus all of them
+02:55 so we're using the count operator here,
+02:57 we can get all the cars by saying card.objects.count
+03:00 or we can run our query and say count and get just the ones that match the query.
+03:03 So in this case, we're going to say you the double underscore yet again
+03:07 this time to use the less than operator,
+03:09 so what we're saying in this case is the bad cars how many are there,
+03:13 well, we're going to go to service history.customer rating
+03:16 and show me all the ones that are less than 4
+03:18 and count how many of those occur.
+03:20 Right, so we'll just use the count operator
+03:23 instead of actually returning deserializing the documents
+03:25 this is much, much faster than saying give me all the bad cars
+03:28 do a lin operator, unless you have a really, really great service.

transcripts/ch7-mongoengine/19.txt

@@ -0,0 +1,71 @@
+00:01 Finally, let's talk about updating documents.
+00:03 It's actually really really easy in Mongo Engine to update document,
+00:06 once you get one back from the database
+00:09 it could be either you've gotten one or you've gotten a whole list
+00:12 and you just happen to be making a change to a particular one of them,
+00:16 it doesn't really matter, so in this case like see in line three,
+00:19 here we're getting a car, we're finding it by id, and we're saying first;
+00:22 first of all, we are verifying that we got the car back;
+00:25 on line five, we're like no, no car, error
+00:28 but let's assume we got the car,
+00:30 we're going to create one of these service records
+00:32 and we're going to append it to the list that is the service history
+00:34 and we want to push that down into the database,
+00:37 we want to save that so all we have to do is call car.save
+00:39 and it will actually push that down.
+00:41 And we saw that there was a possible conflict,
+00:44 a possible raise conditions at the database level
+00:47 if this type of code or changes to other parts of that car
+00:51 that some other operation was being done on the car with the same id
+00:54 it's possible that we could overwrite those changes, maybe, not for sure,
+00:58 depending on how everybody sort of changed different parts of the car,
+01:01 but there could be a problem of saving this here.
+01:04 So you want to careful when you're doing this,
+01:07 but this works totally fine, most of the time, it depends on your situation
+01:11 how actively you're changing, how much contention there is
+01:14 for particular documents, but assuming the contention is low
+01:17 we're going to be able to say get the car,
+01:19 we should make the changes to it and call save,
+01:21 and it'll push that right back to the database.
+01:23 However, if the contention is high, you care about performance
+01:26 or you really just want to take most advantage of MongoDB
+01:30 both for safety and performance, you can use the in place updates.
+01:34 Here you can see we have this owner object that we've introduced
+01:38 and this is like the owner of the car,
+01:41 so maybe we want to record how many times
+01:43 has this owner been to our service shop,
+01:46 owners could own more than one car,
+01:48 and so maybe we want to know like for this particular person
+01:52 they've been in ten times, even though they have a new car
+01:54 that's only been in twice, so we're going to have this number of visits
+01:57 which is an integer on the owner
+01:59 and we can actually use the increment operator right on it like this
+02:03 we can say owner objects id = customer id
+02:06 that's like the primary key, then update one,
+02:08 increment operator double underscore name of the field
+02:10 so incremental_ _number of visits;
+02:12 you can increment it by whatever you want
+02:14 even a negative number which is really a decrement
+02:16 but there's just the increment operator.
+02:18 So basically add that number on the right the one here to the number of visits,
+02:22 so this is cool for the individual operators
+02:25 you can use set, you can use increment, some of those particular operators,
+02:28 but if we're going to work with the set in our case
+02:32 like we just saw we are adding a service record to a car
+02:35 here we could do the same thing,
+02:37 but we could do this in place with the push operator
+02:39 instead of pulling the document back, adding it and saving it again,
+02:42 so we want to create a service record, and this time we're going to say
+02:45 card.objects again the queries the where clause if you will it's id is car id
+02:52 and then we want to say update one and use the push operator
+02:54 so push on to the service history this subdocument.
+02:58 In this case, what we get back is the number of updated items
+03:00 we set up date one so you can bet it's one or a zero
+03:03 and if it's not one something went wrong.
+03:06 So it supports in place updates with the individual value operators
+03:11 like increment and so on, it also supports things like push and add to set
+03:16 for working with sets or arrays in the documents.
+03:20 This is both better in terms of safety because you get this transactional behavior
+03:24 it's also better in terms of performance,
+03:26 because you don't bring the document back
+03:28 make changes and push it in, you just shove the delta down to the server.

transcripts/ch7-mongoengine/2.txt

@@ -0,0 +1,36 @@
+00:01 So here we are at mongoengine.org
+00:03 and MongoEngine is the document object mapper, right
+00:06 they say think ORM but for document databasis, right just like we said,
+00:09 and MongoEngine is a great ODM, which they maybe call it DOM,
+00:15 given their naming, I think ODM is slightly more popular;
+00:17 anyway, it's really great and flexible ODM,
+00:20 it has a very clear way of describing your classes,
+00:24 if you use something like SQLAlchemy and you like the way it works,
+00:26 you really like this, if you like Django ORM it's very similar to that,
+00:30 actually it uses the active record style, not the unit of work style
+00:34 which Django uses active record, so does Ruby On Rails,
+00:37 if you look as opposed to say SQLAlchemy which uses unit of work.
+00:41 It works well in Python 3, it also works in Python 2.
+00:44 So if you go here you'll see there's actually additional things you can get,
+00:48 you can get a Flask plug in on top of this,
+00:50 you can get a Django plug in on top of this ,
+00:52 and some extras as well, there's a couple of cool additions that you get,
+00:56 but we're just going to work with plain MongoEngine,
+00:59 that means we can use it in any application whether it's a web app or not
+01:02 and we can use it however we want in our web application.
+01:06 Like pretty much everything in this course MongoEngine is open source
+01:08 so you can go here to githug mongoengine/mongoengine,
+01:12 you can see it's almost two thousand stars, almost a thousand forks,
+01:15 it was updated fourteen days ago, it's very active and living project,
+01:18 it's one of the things I look for when I depend on
+01:22 some core part of my application is
+01:24 is this thing being updated, is it alive, things like that,
+01:28 you don't want to take on something as critical as your object document mapper
+01:32 if no one is out there maintaining it,
+01:34 you probably don't want to be writing an ODM,
+01:37 you probably want to be using ODM and building whatever it is you're trying to build,
+01:40 like a website or app, or a service api, whatever,
+01:43 it's probably not an ODM you want to be building.
+01:46 So you see, MongoEngine is quite active,
+01:48 and you can go fork it and keep a copy of it for yourself,
+01:50 but for this course, we're just going to pip install it.

transcripts/ch7-mongoengine/3.txt

@@ -0,0 +1,31 @@
+00:00 So far, we've been kind of poking at MongoDB,
+00:03 playing around with some of maybe existing data
+00:05 or creating simple little databases with one or two records in it.
+00:09 We're kind of done with that, we're ready to move on
+00:11 to be building the main application that we're going to build for this course.
+00:15 So we're going to take this concept of a car dealership
+00:18 that does service for autos, sells cars, does service like engine repair,
+00:23 fixes flat tires and so on, for a Ferrari dealership,
+00:26 and that's going to be our demo for the rest of this course.
+00:29 On this first go round, we're going to start out
+00:32 with an empty database or a non-existent database,
+00:34 we're going to model it in MongoDB with MongoEngine
+00:38 and then we're going to run that code and create a few simple cars,
+00:41 a couple Ferraries, maybe associate the cars with some owners,
+00:45 do some service on the cars, somebody over rev the engine
+00:48 and has got to get a new engine, or got a flat tire,
+00:51 things like that; we'll see how it all works.
+00:53 Later, when we get to the high performance section,
+00:55 we're going to instead of start with an empty database
+00:57 start with one with like a quarter million cars and tons and tons of service records
+01:02 and they will start asking really interesting questions
+01:05 and really focus on the performance side of things.
+01:08 So we're going to use this for the rest of our time
+01:10 and I have been really waiting till we got to the MongoEngine section
+01:13 to create what I would think of as a somewhat realistic complex demo
+01:17 because with PyMongo it's fine, but you'll see the real power of modeling this
+01:21 in a full featured realistic production style way
+01:25 once we get to MongoEngine things like indexes and uniqueness,
+01:29 and constraints and types and lots of good stuff.
+01:32 So I hope you're ready to learn MongoEngine,
+01:34 and put it to work building this cool Ferrari dealership.

transcripts/ch7-mongoengine/4.txt

@@ -0,0 +1,61 @@
+00:01 Here we are in the github repository for the course,
+00:03 now notice I put the PyMongo play around stuff that we did
+00:06 into a folder called dir 5 PyMongo,
+00:09 now, we're over into our MongoEngine section,
+00:12 and there's actually two things here,
+00:14 there's a service central starter and then there's a server central;
+00:18 so a lot of times people like to follow along with the code examples
+00:20 which I totally encourage, and this one is the way,
+00:23 it when we saved in the repository exactly the way we're about to get started.
+00:27 This one we're going to evolve throughout this demo
+00:30 until it becomes sort of the final version,
+00:32 so I want to open this in PyCharm, and I want to use a virtual environment to do that,
+00:37 so there's a couple of cool tricks I could do to make a life as easy as possible,
+00:40 so here I am in that service central place, and if I do an ls
+00:47 even pin files you see there's nothing other than
+00:49 this sort of starter Python ting we'll talk about in a minute.
+00:51 So the first thing I want to do is I want to actually set up a virtual environment
+00:55 with Python 3s venv, I'll do a dash copies , and I'm going to call .env,
+01:00 and the name .env here is something that PyCharm will look for,
+01:03 so if I open this in PyCharm, after doing this
+01:06 it will actually automatically use this virtual environment,
+01:09 so that's cool, that'll save me a few clicks.
+01:12 Let's go over here and throw this in PyCharm,
+01:16 now it's going to take it a second, it's sort of looking
+01:19 through that virtual directory, let me add the source control.
+01:26
+01:28 So here's a really simple starter application that we're going to talk about
+01:31 but first let's make sure that we have PyMongo installed.
+01:34 So let's just do a quick list, and notice
+01:37 we're already automatically using our virtual environment,
+01:41 that's because it's top level the project, and it's named .env
+01:45 so PyCharm said cool, we'll use that, I didn't have to do anything
+01:48 that's why I did that first thing in the terminal before open in here.
+01:50 So notice we have basically nothing,
+01:53 probably worthwhile to upgrade setup tools,
+01:59 some of the things that depends on C completion sometimes a little nicer,
+02:04 if I have that set up, ok so now we can pip install MongoEngine
+02:08 and you'll see that also it's going to install PyMongo,
+02:11 depends on 2.7.1. or greater, and it's thinking about PyMongo,
+02:15 thinking about MongoEngine, and then we'll be done.
+02:19 Perfect, it also uses six for Python 2, Python 3 compatibility.
+02:24 All right, so now we have our system all set up, we have PyMongo installed
+02:29 and here let me just show you this super simple little app,
+02:32 there's absolutely no MongoDB stuff going on,
+02:34 so we have this main that is going to print this header,
+02:37 very cool, you can see we're going to call our app service central
+02:41 and it's going to do this user loop, and the user loop just says
+02:46 here's some actions that you can do, you can add a car,
+02:48 you can list cars and if you look at implementation,
+02:52 all of these are entirely empty,
+02:54 here is where the MongoDB stuff is going to be happening,
+02:56 so let's go and run this, notice there's no run configuration over here
+02:59 no green button to run, so I can right click on this and say run
+03:03 and it runs and actually let's make this little higher,
+03:07 notice that it's running with the Python out of our virtual environment, Python 3,
+03:11 okay, and now here are little header and then here's our user loop,
+03:15 it says you can add a car, cool to do add a car,
+03:17 you can list the cars, you can find a car, you perform service,
+03:21 right so we just basically have the structure in place
+03:23 and we're going to use this for the rest of this demo,
+03:27 and like I said, we're going to be building on this concept of what we create here.

transcripts/ch7-mongoengine/5.txt

@@ -0,0 +1,85 @@
+00:01 Now let's begin by setting up MongoEngine,
+00:03 there's a few start of the app kind of configuration things we need to do
+00:06 in order to use MongoEngine, and then we just use
+00:10 the classes and types throughout the app.
+00:13 So what I want to do is I'm going to create a folder here
+00:16 let's call it NoSQL, so we're going to put
+00:18 a number of MongoEngine related things in here
+00:22 and I don't want to call it MongoEngine because then it will conflict with the name
+00:26 so, lacking creativity I'm calling it this,
+00:29 now there's a couple of things we need to do
+00:30 we need to set up the connections and then we need to define the classes,
+00:34 this first part we're just going to set up the connection.
+00:39 I'll create a module called Mongo setup, ok
+00:43 so down here, let's define a function
+00:49 called global init, we are going to call this function from outside.
+00:53 Now, in real life later as we talk to like sort of the production stuff
+00:57 we're going to want to pass in like the user name, the password, the server name
+01:00 all sorts of stuff that you know maybe in a real app comes from
+01:04 like a config file or the environment in a production server, something like that,
+01:07 but for now we're just going to put this in here.
+01:10 So to get started, we have to import MongoEngine,
+01:14 we don't need PyMongo but MongoEngine we need.
+01:16 And then down here, it's really simple what we need to do,
+01:19 we're going to register a connection,
+01:22 so we're not actually going to open the connection here,
+01:24 this doesn't talk to the database, but it basically says
+01:26 look if you have a class that maps
+01:30 to a particular type or named part of our application
+01:34 use this database connection to do the backend work.
+01:37 So we're going to come down and say Mongoengine.register connection
+01:40 and see it has alias name and then other,
+01:42 and what comes with the other, the .... there
+01:46 is like the connection string information
+01:48 like server name, port name, host name, use ssl, replica set, all that kind of stuff.
+01:52 Okay, so we're going to say, make it really explicit here
+01:56 we're going to say alias, I was going to call this core
+01:59 and I'll you what that means in a minute, so let's call this demo_dealership.
+02:04 Now normally, I would probably just use dealership
+02:07 but I already have that for something else
+02:09 in a previous example, I kind of want to keep it around
+02:12 so we're going to say demo_dealership,
+02:16 there we go, and that's all we're going to need to do.
+02:18 So the idea is here, we could have multiple things like analytics
+02:22 it could be here, and this could be visits or whatever,
+02:26 it could be mapping to another database assuming I spelled analytics correctly ;
+02:31 so in our classes, we can say this class belongs in the core database,
+02:34 whatever that happens to be configured as,
+02:37 this one over here, happens to belong in the analytics database
+02:40 and so I find it's really valuable if you've got like some core data
+02:44 that are required to make your app run,
+02:46 and then like huge amounts of extra analytical type data,
+02:49 that if you lost, it's like oh well I'd rather have that data
+02:52 but if for some reason I want to back up
+02:54 let's say you've got 5 GB of analytic data and a 100 MB of core data,
+02:59 you could run backups on the core server
+03:02 much more frequently than the analytics one
+03:04 and by partitioning them to different databases or even different servers
+03:07 you can do a lot of cool tricks like that.
+03:10 Alright, all that said, we're not doing that,
+03:12 we're just going to have one database that we're calling core
+03:15 so we're going to register this connection
+03:17 and when we get to defining the classes
+03:19 you'll see a place where we refer to the core connection,
+03:21 that's what we've configured here, and it's going to default local host
+03:25 default port everything like that.
+03:27 Again, when we get to the using MongoDB in production,
+03:30 we're going to talk about how to pass all the extra information you need
+03:33 to use this for real, on another server, on another port,
+03:36 with authentication, everything,
+03:39 but for now, this is what we're going to do to set it up.
+03:41 So let's go ahead and get started, using this,
+03:43 let's go down here, and we've got our print header,
+03:46 let's go ahead and do a config Mongo,
+03:50
+03:59 so it's easy enough to import, let's go up here at the top, our module,
+04:03
+04:09 so we'll just call it Mongo setup like this, and I'll just say global init
+04:13
+04:18 do a little pep8 formatting, and we're good to go,
+04:20 and it thinks this is misspelled, no, just short alias for MongoDB.
+04:25 Okay let's just run it to make sure everything is working,
+04:29 alright, there is no real way to test it yet, but in a moment, we will,
+04:32 so far everything worked, we configured our MongoDB connection,
+04:35 next up, it's to actually think about modelling these cars
+04:38 and owners and service, and all those kinds of things.

transcripts/ch7-mongoengine/6.txt

@@ -0,0 +1,145 @@
+00:01 Alright, let's start defining our classes that we're going to map to the database.
+00:04 And I guess the first place to begin would be to describe
+00:06 how I think we're going to store the data and what the data is;
+00:09 so we're going to have a car, a car is going to have an engine,
+00:14 with lots of details about the engine like its horsepower and so on,
+00:17 a car is going to have a service history and each entry in the service history
+00:21 is going to be some additional information, like what was the work performed,
+00:24 how much did it cost, when was it done, that kind of stuff.
+00:28 There is going to be an owner who can own multiple cars
+00:32 and a car can be owned by multiple people,
+00:35 so there's a many to many relationship between owners and cars,
+00:38 and then owners have personal information like their address and stuff like that.
+00:42 So really the idea is we have cars and owners,
+00:45 and then the cars have additional things like engines
+00:47 and then the thing you can do to the car that's really interesting is
+00:50 you can give it service right, change its tires,
+00:53 change its parts and plugs, give it a new engine and so on.
+00:55 So we want to model those things, so let's start right at the heart of it,
+00:59 let's start with the car.
+01:01 So over here, we're going to define another class,
+01:05 another Python file called car,
+01:09 and we'll go down here and we're just going to define a class called car, like this.
+01:15 Now, we're going to need to work with MongoEngine,
+01:19 because the way it works is all the classes, all the entities
+01:22 we want to map to the database are going to derive from mongoengine.Document;
+01:27 now this allows us to load and save and query the documents,
+01:31 it also provides a field called id, which maps to underscore id in the database
+01:40 and by default is an object id type of thing,
+01:44 okay so we don't have to worry about this id
+01:46 whether it's an object idea or not, you can change it
+01:48 you can put a different one and overwrite it,
+01:50 but if you leave it alone this is what you get.
+01:52 Okay, so the car now has an object id and we're going to give it
+01:55 a couple of pieces of information like about what model is it,
+01:58 so if you've worked with these ORMs before, they are very similar,
+02:02 what we're going to do is we're going to define the properties of the document
+02:06 as basically a descriptor, so it's a mongoengine.
+02:09 this is going to be a string, so we'll say string field
+02:12 so you have sorted list field which is pretty sweet,
+02:14 we're going to start with a string field
+02:17 that's nice and easy, let's add while we're at it a make
+02:20 so a model might be F40, make would be a Ferrari,
+02:24 we're going to have a year, mongoengine.IntField
+02:28 now notice, we have types here, we have strings and we have integers
+02:32 in MongoDB, things have a type in bson they're strings or they're integers
+02:36 but there is no way to enforce a type, there's no way to say
+02:39 the year must be an integer you could easily make it a list if you want it,
+02:42 make anything you want, but in MongoEngine, it has a concrete type
+02:45 which is actually really valuable.
+02:47 Let's have a mileage, and let's say the mileage is going to be a float field
+02:53
+02:56 and then it's going to have a vin number, vehicle identification number
+02:59 and that is going to be a mongoengine.StringField
+03:03 because it might have alpha numeric in it, it might start with zero, things like that.
+03:07 Okay, so this pretty much is what we got to do in order to map this to the database.
+03:12 However, there's one more thing that you want to do,
+03:15 so we're going to define this meta dictionary
+03:20 and the dictionary is going to say the database alias we want to use is core,
+03:26 remember that from over here, we said this connection to this database
+03:31 with all the properties that we're not specifying because they're defaults
+03:33 but we could have a server name, port authentication, all that kind of stuff
+03:37 we're going to say go find this connection that we've registered here
+03:40 because the db alias we want to use is core;
+03:43 I find this a really nice way to partition our app up into like central parts
+03:48 and analytics and reporting and those kinds of things.
+03:51 Then we can also control the name of the collections,
+03:54 we don't want to be capital C Car, how about lower case cars.
+03:57 Alright that's more pythonic for us,
+03:59 so we're going to call our collection cars,
+04:01 and in the shell we would say db.cars.find,
+04:04 alright, but here we're going to work with MongoEngine.
+04:07 So this is not the end game, but this is definitely the beginning,
+04:10 let's go down here and write some throw away code
+04:13 just to see that we have everything hanging together.
+04:15 So let's go down, hit command b, so go to add car,
+04:18 and let's see what do we need here,
+04:22 let's go and grab the stuff we're going to need,
+04:24 in fact, you'll see that some of these we're not going to have to set
+04:27 especially when we get to the final version of what car looks like,
+04:33 but let's say we want to get the model, it's going to be input what is the model
+04:38 I could almost just enter Ferrari because that's what it always is the make,
+04:42 so we have to ask the user some questions here
+04:47 and I'm going to assume this is going to work,
+04:51 assuming that we can parse that as an integer
+04:59 and here we'll say mileage, that's going to be a float
+05:04
+05:08 and let's go and get a vin number.
+05:11
+05:18 Okay, so now we want to create a car we want to insert it into the database
+05:22 and later maybe even do a query with it, so we'll say car = Car like this
+05:25 and I could use keyword syntax to set the value here
+05:28 let's go ahead and import that to the top,
+05:32 so I could say model equal such and such, year equals each and such.
+05:35 Or I could say car.year = year, card.make = make,
+05:43 notice the auto complete which is very nice,
+05:45 model and we'll just keep going like this.
+05:49 And then, in order to insert it, all we have to do is go to the car and say save
+05:54 this is the active record style, in active record you work with a single document
+05:58 and you say load, save, query, things like that right,
+06:02 you save them individually, which maps really well to MongoDB
+06:05 because it doesn't have concepts like transactions.
+06:09 So let me just put in something wrong here for the mileage,
+06:12 remember the mileage, if you look over here, has to be a float
+06:16 so let's try to put a string in there, all right so run my thing,
+06:20 I want to add a car, it is not going to make it through I believe,
+06:23 so let's say 1998, it's going to be abc— it's going to crash,
+06:33 and it says car validation error no, no, no,
+06:37 the mileage only accepts floats and integers
+06:39 so already in the simplest form of our car, I'm going to do a lot more to it
+06:43 it's already helping us out a lot here,
+06:45 so oh yeah yeah yeah that was supposed to be a float,
+06:48 do you know how easy it is to make that mistake
+06:50 when you're working in raw dictionaries and put in a string in the database
+06:52 when it should have been a float, and then how do you do a sort,
+06:54 how do you do a greater than— you're out of luck, right
+06:57 so we already get a huge value from like the simplest variation.
+07:00 Okay, let's go and put this in for real now, add a car,
+07:03 it is a F40, it's a Ferrari, it was built in 2005,
+07:07 this time the mileage is 10 thousand miles, and the vin is af2.
+07:15 There we go, ready?
+07:18 Oh it looks like I made a small mistake configuring MongoEngine here,
+07:22 let's go back really quick, that's unfortunate,
+07:24 so over here if you look, I quickly typed in the alias and I said db
+07:29 but no no, the thing that we want to use, the name of the database is not db its name,
+07:33 so sorry about that, let's fix this here,
+07:36 all right, now let's do it again, now we should be able to add our car
+07:40 which we're going to go over here, we don't really need this anymore,
+07:47 so we are going to ask for the input from the user, create the car and save it.
+07:53 Now that it actually knows what database to use, that should be pretty easy.
+07:57 So add the car, F40, Ferrari, 2005, driven slightly further since we crashed it
+08:09 and we tried to add it but here we go,
+08:11 and this would be a F20, boom just like that,
+08:16 we've added it, remember, this demo dealership didn't even exist
+08:20 until we just hit enter there, now let's see what we got,
+08:24 go back over to our favorite shell RoboMongo,
+08:27 and now we have dealership which was already there
+08:30 but now we have demo dealership, and check this out
+08:33 we have cars and in here if we look at it like this,
+08:35 there we have our model, our make, our year, our mileage and notice,
+08:39 this is an integer, this is a float—
+08:42 why, because in the class that's how we defined it
+08:44 and the other two obviously are strings, here is the underscore id
+08:47 that was generated or brought into existence
+08:50 by being just mongoengine.document class,
+08:54 we didn't have to do anything to make that work.

transcripts/ch7-mongoengine/7.txt

@@ -0,0 +1,69 @@
+00:01 So we are able to create a car and this is not a great architecture
+00:04 to just jam the writing here, but for now
+00:06 we're just going to leave it right into our main application.
+00:08 However, let's go look at the car definition again,
+00:11 there's a couple of things that would be nice, it's not part of MongoDB
+00:14 but it would be nice to require like we already have our type checking,
+00:17 it would be nice to say that you have to specify a model and a make
+00:21 and it would also be nice to say you have to specify a year,
+00:25 but maybe we could have the mileage default to zero,
+00:29 like 0.0 as a float for example
+00:31 and it would be even cool to generate the vin id number here for new cars, right.
+00:36 Typically that comes with the car automatically
+00:38 and you don't have to worry about it, you have to know what your id is.
+00:42 So it turns out that this is super easy and this only is available in MongoEngine,
+00:47 it is not available in PyMongo and it's not available in the database itself.
+00:50 So we can come down here we can say this is required
+00:53 or must match a regular expression, or whatever,
+00:56 so we're going to say required is true,
+00:58 so you must specify a model and given this is a Ferrari dealership
+01:02 we could either say this is required or we could give it a default Ferrari,
+01:05 it's going to just make it required; the year also is going to be required
+01:10 so you have to type those three things in,
+01:13 but maybe over here we could have the default be zero.
+01:15 New cars have 0.0 miles, that seems fair,
+01:19 how about this, how about auto generating that,
+01:21 well, the default is really callable and we could just put actually like this
+01:25 we could say float and it should call the constructor initializer for float
+01:30 we also put a value, so if we go up here, we can use uuid,
+01:38 so if we import that, go have a quick look,
+01:44 you can use uuid4 as a string
+01:47 so if we say stir of this, we have that dash,
+01:51 I don't think vin numbers have the dash
+01:53 so we could replace dash with nothing like this,
+01:56 what do you think that for vin number,
+01:58 so if we could get this little bit of code to run
+02:00 on every time we insert a new thing, hey that would be cool, right,
+02:04 and we can, so we go over here and say default,
+02:06 I would like to call a function that returns that value,
+02:09 the simplest way is to do a lambda that takes no parameters and returns that, ok.
+02:13 So that's cool, let's actually wrap this around
+02:16 so it fits a little better on the big font, small resolution,
+02:20 so now we have a better version, let's go back here
+02:22 and now we can take away some of these things
+02:25 that we can just let it get generated and we'll save it
+02:28 so let's try this one more time, so I'm a going to go down here and say add a car
+02:31 the model is F40 again, and Ferrari, the year is shiny new, 2017,
+02:40 boom, notice it didn't ask me about the mileage or the vin number,
+02:44 but did that work, let's go find out, open the shell, turn it again
+02:49 and look at the bottom one, check out the vin number, how cool is that.
+02:52 So we've got our vin number down here,
+02:54 right this is the one I said 2017 not 2005,
+02:57 this was generated by our default value, this was generated by our default value,
+03:01 and if I haven't done it yet, but if I for some reason omit setting the make
+03:06 let's see what happens if i don't set the make, remember it's required
+03:09 it doesn't matter what I put here, boom field is required, make,
+03:13 right, we can't save it without setting the make
+03:16 but we can save it without setting the vin number because that has a default.
+03:19 Okay, so go back here, so we can use required and default value
+03:24 as well as other types of things like regular expression matches,
+03:28 these default values can be straight up values or they can actually be callables
+03:32 which result in returning the thing that is the default value
+03:35 in this case each time we add a new car,
+03:38 maybe I'll show you, we'll get a new one,
+03:41 it's going to be 308, and it will be a Ferrari,
+03:44 and it's going to be built in what is that, 1991,
+03:47 now if we go look one more time, there is a 308 again
+03:52 totally distinct number or vin number here, right
+03:55 because each time we insert it, it's calling that lambda,
+03:58 each time you call the lambda, you get a dramatically different uuid.

transcripts/ch7-mongoengine/8.txt

@@ -0,0 +1,108 @@
+00:01 So we have the primary properties of our car modeled,
+00:04 they have their required fields, they have their default values, things like that.
+00:08 We still maybe want to consider indexes
+00:10 but we're saving that for a performance area.
+00:12 The next thing we want to look at is the engine, and the embedded elements.
+00:16 We talked about at the beginning that the car is going to have an engine,
+00:20 and it's going to be equal to something,
+00:22 not a string or a float or something like that,
+00:25 but in fact, to an entire subclass, right,
+00:28 a class that represents engines in particular,
+00:30 so let's create that class and then we'll come back to the car.
+00:33 So if we come over here, I'm going to create something called engine
+00:37 and just like before, we are going to import MongoEngine,
+00:41 not the same engine, right, class engine
+00:44 and this is going to derive from MongoEngine document,
+00:46 now, before I said document has the id
+00:49 and this is like the top level thing that allows saving and loading,
+00:51 so we don't use this type for embedded documents, subdocuments,
+00:56 right, subdocuments don't necessarily have ids,
+00:58 you don't load query and save them etc independently,
+01:02 you can only work with them through their parent document,
+01:05 so in this case, we are going to say this is only allowed as an embedded document
+01:08 it can't be queried or saved directly, but it can be used
+01:11 as a subelement of another type, like for example our car.
+01:17 So let's go over here, and give our engine a couple of properties,
+01:20 we're going to give it the horsepower
+01:23 and the horsepower is going to be a mongoengine.integer,
+01:28 so this is going to be an int field, it is going to have a leaders
+01:31 so the size of the engine and this will be mongoengine float field
+01:35 because it could have like 2.3 liter engine something like that;
+01:40 we'll have the mpg, so miles per gallon,
+01:44 and this is going to be the same thing, a float,
+01:46 finally it'll have a serial number, and this is going to be a mongoengine string field.
+01:55 And the serial number is kind of like the vin number,
+01:58 but in fact it's not exactly the same, it's going to be having dashes
+02:03 it will have a slightly different format, but let's go ahead
+02:05 and work on some of the default values in that,
+02:08 so we're going to import uuid, I am going to use just again uuid
+02:12 to actually generate this so quick review, default is a lambda,
+02:16 and the lambda is going to return a string representation of uuid4,
+02:22 I believe the dash is in there this time, just to make sure
+02:24 hey this is clearly not a vin number, it's a serial number.
+02:28 And let's set these all to be required, so we can have in the subdocument itself
+02:34 these required values, except for the serial number,
+02:36 which is going to be autogenerated;
+02:38 all right, so let's go back now that we have this not a document
+02:41 but a subdocument, an embedded document, let's go back to the car
+02:45 so in the car what am I going to set this to, a MongoEngine something,
+02:48 right so this is not going to be a string or float or anything,
+02:52 it's going to be an embedded document field,
+02:54 right, so we have an embedded document list field or just a field
+02:58 so this is a single engine, not a list of them, so here it goes like that
+03:02 and then I need to tell it not just what goes in there, but the document type
+03:06 what is the subdocument, the subdocument is actually this,
+03:09 and then let's go ahead and say required = true
+03:12 so we could even say that this subdocument cannot be
+03:15 none or null in Javascript, it has to be set to a thing.
+03:18 So, we're going to import the engine, so it knows the car,
+03:22 it knows about the engine and it can save it there,
+03:25 all right, let's try to work with our engine here and see what happens.
+03:28 Okay, it runs, that's already pretty encouraging,
+03:31 so the model is going to be the Testarossa,
+03:34 I'm sure that's misspelled but we'll roll with it,
+03:36 Ferrari, it was built in 2010 and this is going to crash
+03:41 because the engine was not specified, how do we do that,
+03:44 well let's jump right where the problem is and find out.
+03:48 So we need to set the engine, now I'm just going to hardcore engine setting
+03:52 so we don't ask this anymore, right, so we want to come over and say
+03:56 allocate an engine, allocating it is just the same as the top level item
+04:01 we'll say engine.horsepower is around six hundred horsepower,
+04:04 that's pretty insane isn't it, we have the miles per gallon,
+04:08 I think that's around 20, not super high, liters let's say 5.0
+04:17 it's not exactly right, I'm sure but close enough;
+04:20 and then we just say car.engine is engine, like this,
+04:24 so we create this object and we associate it,
+04:26 and then later we can say car.engine. and we get all the various things here.
+04:30 Okay, so here we have our car, we set the engine and now let's do this again;
+04:35 in fact, let me just comment this out a little bit,
+04:40
+04:42 yeah, we'll just ask those two questions, keep it little simpler.
+04:45 Ok so we're going to add a car, the model is Testarossa,
+04:49 don't think we have one yet, let's open up our shell,
+04:53 we have the couple F40s and the 308, but no Testarossa,
+04:59 2005, and boom, inserted, okay let's run this again.
+05:06 Where did it go, oh there it is, check that out how awesome is that!
+05:10 So we have our Testarossa 2008, the mileage defaulted to zero,
+05:14 the vin number was autogenerated, now here we have our engine
+05:18 and check this out, here is our subdocument
+05:21 curly brace means object subdocument in json
+05:24 so we have horse power 600, liters are 5,
+05:26 miles per gallon is 20, serial number is such and such,
+05:30 let's go and make a quick change here so we can ask some interesting questions,
+05:34 let me make this 22, this is a more efficient version
+05:38 and it only makes 590 horse power, okay let's just insert one more.
+05:43 So we're going to add what model, so let's say 355
+05:47 is that a thing, I'm not sure, 2000, like so.
+05:51 So if we go over here and run this, now we have these two
+05:57 that have engines, this one has a 590 and so on,
+06:00 so we can actually go over here and ask interesting questions
+06:03 like I want all the cars with an engine where,
+06:06 let's go for liters, is we can say something like $ > what value, say 5.95
+06:15 and of course, got to close everything off,
+06:19 what did I miss— I missed something didn't I, because it didn't come back,
+06:24 horse power could be that much or liters could be something much smaller
+06:28 so here, horsepower this much, or liter, I could have done 4.5,
+06:33 there you go, so now you can see that we can query down into this subdocument
+06:37 but it's going to get more interesting when we start doing queries with MongoEngine,
+06:40 because we want to get these rich objects back.

transcripts/ch7-mongoengine/9.txt

@@ -0,0 +1,84 @@
+00:01 Now that we have our engine stored in our car
+00:04 over here as an embedded document,
+00:06 the next thing that we need to work on in our car model
+00:08 is how do we store the service histories,
+00:11 first of all, what kind of data are we going to have in the service history itself.
+00:15 So let's go create a class that represents that
+00:17 and then we'll figure out what to do with it.
+00:20 Again we're going to import MongoEngine, create a class called service history
+00:27 and we're going to postpone discussing what goes in there for a minute.
+00:32 So this is going to have a date, where this has happened
+00:36 either like a create a date or the date when the service was done,
+00:39 so let's create a MongoEngine, a date time field and let's even set the default to now,
+00:45 so we'll go over here and say—
+00:48
+00:53 so we want to set this to be a lambda,
+00:55 actually we don't need to set it to a lambda,
+00:57 we'll just set it to datetime.datetime.now without parenthesis,
+01:01 we don't want to call it, we want to just pass that function,
+01:04 so we're going to call the now function whenever something gets inserted
+01:07 so the date we could even call this like service date if we want,
+01:10 but I'm going to stick with date.
+01:13 The next thing is let's have some kind of description here
+01:15 like just some text, we'll say description,
+01:18 it's going to be a mongoengine.StringField,
+01:22 and it is just going to be like changed the oil, they had a flat tire,
+01:26 a nail was stuck in it, we patched the tire and everything was good,
+01:30 something like that, right super simple;
+01:32 we have a price, this is how much they paid for the service,
+01:35 so it will be a float field, and lastly we care about our customers
+01:38 we're primarily a service shop and sometimes we sell our Fearraries
+01:43 and sometimes we just service them, but we want our customers to be happy
+01:47 and how do you know whether they're happy— we better ask them,
+01:50 so let's ask about their customer rating
+01:55 and this is going to be an int field, so we're going to set this
+01:58 this number is going to go from let's say one to five
+02:03 five being the most satisfied, one being the least satisfied.
+02:07 Great, so now here's the deal, do we want to embed this into the car
+02:12 like we did the engine or do we need to come over here and say something like this
+02:17 car_id = mongoengine.ObjectIdField, like this, right
+02:23 so we're going to have a relationship over to the car
+02:25 or maybe the other way around, on the car, we could have some bunch of ids
+02:31 that represent the service history, or there's a bunch of other options.
+02:33 So remember when we're designing our documents
+02:36 one of the primary questions is in our application
+02:39 do we want that embedded data with us, most of the time
+02:43 and it turns out because almost all of our data work
+02:47 are reason to pull up one of these cars is to actually look at the history of it
+02:51 we are going to decide for that reason, that we do
+02:54 almost always want the service history associated with the car
+02:57 and we don't usually need the service history without the car,
+02:59 we need details about the car like the mileage for example.
+03:02 How are we going to do all that— let that means
+03:06 we probably want to embed the service history as an array into this car;
+03:10 the other thing we have to care about is
+03:13 is that set bounded and is that bound small?
+03:15 You know, a car how much could it possibly get worked on,
+03:18 right let's say it gets worked on once a month every month, just ongoing,
+03:24 very very unlikely, but that would give us at most a hundred of these service histories
+03:28 let's say for some reason that like that upper bound is totally fine with us,
+03:32 it's certainly not crazy unbounded where it's going to escape the 16 MB ram
+03:36 I mean, how much could one these service histories be, 2 K each, not a huge deal.
+03:40 So for all those reasons we are deciding to embed the service history into the car
+03:45 so we want to come over here just like we did with engine
+03:47 I'll say service history, could be plural could be singular, let's go with singular
+03:51 so I'm going to go mongoengine, now it's not an embedded document field
+03:54 because this is not a single document, this is a list,
+03:57 so instead, is we are going to have an embedded document list field,
+04:01 now over here in this one, we said what is the type that is the embedded document
+04:05 here what we can say is what things, what type of things are contained in that list
+04:10 so this will be a service history, we've got to import that, thank you PyCharm,
+04:16 and then we could come over here and we could say
+04:19 the default value is like a new array or something, a new list
+04:22 but in fact, that's what it already is,
+04:24 so the default value for embedded document list is a new list for each entry,
+04:29 so we're just going to go with that, that seems totally good to us.
+04:32 All right, so now we have this mapped, let's actually go back to our app
+04:36 and add the ability to create and insert some histories.
+04:40 One thing that we almost forgot, since we decided
+04:43 we're going to embed this service history,
+04:46 that tells us how we need to treat the base class for the service history.
+04:49 So recall for the embedded items,
+04:52 this is going to be a mongoengine.EmbeddedDocument,
+04:55 if it was going to be standalone and in its own collection it would just be a document.
+04:59 There we go, now our service history is ready to go.