From 6c0f760eb3463e0372575be0e7caf019389f51fe Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Mon, 25 Sep 2017 10:27:47 -0700 Subject: [PATCH] Transcripts for first three chapters of MongoDB course. --- transcripts/ch1-welcome/1.txt | 38 ++++++ transcripts/ch1-welcome/2.txt | 29 +++++ transcripts/ch1-welcome/3.txt | 97 ++++++++++++++++ transcripts/ch1-welcome/4.txt | 25 ++++ transcripts/ch1-welcome/5.txt | 15 +++ transcripts/ch2-what-is-nosql/1.txt | 48 ++++++++ transcripts/ch2-what-is-nosql/2.txt | 38 ++++++ transcripts/ch2-what-is-nosql/3.txt | 80 +++++++++++++ transcripts/ch2-what-is-nosql/4.txt | 85 ++++++++++++++ transcripts/ch2-what-is-nosql/5.txt | 58 ++++++++++ transcripts/ch2-what-is-nosql/6.txt | 106 +++++++++++++++++ transcripts/ch3-computer-setup/1.txt | 89 ++++++++++++++ transcripts/ch3-computer-setup/2.txt | 167 +++++++++++++++++++++++++++ transcripts/ch3-computer-setup/3.txt | 45 ++++++++ transcripts/ch3-computer-setup/4.txt | 69 +++++++++++ transcripts/ch3-computer-setup/5.txt | 24 ++++ transcripts/ch3-computer-setup/6.txt | 82 +++++++++++++ transcripts/ch3-computer-setup/7.txt | 23 ++++ 18 files changed, 1118 insertions(+) create mode 100644 transcripts/ch1-welcome/1.txt create mode 100644 transcripts/ch1-welcome/2.txt create mode 100644 transcripts/ch1-welcome/3.txt create mode 100644 transcripts/ch1-welcome/4.txt create mode 100644 transcripts/ch1-welcome/5.txt create mode 100644 transcripts/ch2-what-is-nosql/1.txt create mode 100644 transcripts/ch2-what-is-nosql/2.txt create mode 100644 transcripts/ch2-what-is-nosql/3.txt create mode 100644 transcripts/ch2-what-is-nosql/4.txt create mode 100644 transcripts/ch2-what-is-nosql/5.txt create mode 100644 transcripts/ch2-what-is-nosql/6.txt create mode 100644 transcripts/ch3-computer-setup/1.txt create mode 100644 transcripts/ch3-computer-setup/2.txt create mode 100644 transcripts/ch3-computer-setup/3.txt create mode 100644 transcripts/ch3-computer-setup/4.txt create mode 100644 transcripts/ch3-computer-setup/5.txt create mode 100644 transcripts/ch3-computer-setup/6.txt create mode 100644 transcripts/ch3-computer-setup/7.txt diff --git a/transcripts/ch1-welcome/1.txt b/transcripts/ch1-welcome/1.txt new file mode 100644 index 0000000..c901ab1 --- /dev/null +++ b/transcripts/ch1-welcome/1.txt @@ -0,0 +1,38 @@ +00:01 Hello and welcome to MongoDB for Python developers. +00:04 Throughout this course, we're going to learn +00:07 how to connect, model for and build applications with MongoDB, +00:11 and we're going to do this with Python. +00:14 We're going to look at the straightforward, lowest level way of doing things +00:17 with PyMongo, and we're going to look at mapping classes +00:20 through what's called an ODM, think orm for Mongo; +00:24 an ODM to Mongo DB with Mongo Engine. +00:28 And these come together to make a great combination. +00:31 So let's begin by talking about document databases, how do they work? +00:35 Well document databases in some ways +00:38 are very much like standard relational databases, +00:40 they have what you would think of as columns in that relational world, +00:44 title, course id and duration in seconds here for example, +00:48 but it also has nested data, so in a relational database +00:51 we might have a lectures table that has +00:54 some kind of foreign key constraint back to a chapter +00:56 in this example we have on the screen here, +00:59 but in fact in a relational database, we can embed those lectures +01:02 inside of the chapter object, why is this good? +01:06 Well, often, we spend so much time and energy +01:09 building up an object hierarchy in our application +01:12 and then tearing that apart into a bunch of little pieces +01:15 what's called third normal form, basically normalizing our data +01:18 in a relational database and then building a backup, taking it back apart +01:22 and this object relational impedance mismatch +01:25 makes it hard for us to reason about our code, +01:28 it makes it a little bit less intuitive and so on. +01:31 So with document databases, we can model our data +01:34 the same way our application wants to use that data. +01:37 We also have more flexibility in the schema, +01:39 which means that deploying a new version of our application +01:43 often does not require some kind of migration script and downtime, +01:47 no, we just deploy the new version and the document database adapts to it. +01:51 So document databases in my opinion are the best way to build applications, +01:55 for the 80 percent case right, maybe you have some edge case +01:58 where that doesn't make a lot of sense +02:01 but most apps really benefit from using document databases. \ No newline at end of file diff --git a/transcripts/ch1-welcome/2.txt b/transcripts/ch1-welcome/2.txt new file mode 100644 index 0000000..0414087 --- /dev/null +++ b/transcripts/ch1-welcome/2.txt @@ -0,0 +1,29 @@ +00:01 Which one happens to be the most popular, the most widely used? +00:04 Well, you can guess, it's probably MongoDB, given this course, right. +00:07 However, you probably didn't guess how much more popular MongoDB is +00:13 relative to its other NoSQL friends. +00:17 So we've got CouchDB, this orange one, +00:19 which is way down there, and sort of not even trending; +00:22 well, we've got RavenDB which is basically not used, +00:25 Cassandra, which kind of peaked around 2016, is heading down. +00:29 We got MongoDB just much, much more popular than these, +00:34 if you want to experiment or play around with this data yourself, +00:36 you can check out the link at the bottom. +00:39 So MongoDB is really, really popular, +00:41 it's by far the most popular, widely used document database. +00:44 It's also really loved, so one of my favorite places +00:47 to get insight into to the developer community is Stack Overflow's yearly survey; +00:52 so on the 2017 yearly survey, you can see +00:56 MongoDB appears very high up the rank for most loved technology, +01:01 and if you look at the little description at the bottom, +01:03 this represents the number of developers +01:05 who are currently using the technology and really enjoy working with it. +01:09 So MongoDB is right up there, among all at the top. +01:12 Now, what about wanted? +01:15 In fact, MongoDB just dominates the wanted category, +01:17 so this is people who are not currently using MongoDB +01:20 or currently using whatever technology is listed here, +01:23 but they wish they were, right. +01:26 So MongoDB is definitely highly, highly desired +01:29 for the people who are not using it, +01:31 and very much loved by the people who are. \ No newline at end of file diff --git a/transcripts/ch1-welcome/3.txt b/transcripts/ch1-welcome/3.txt new file mode 100644 index 0000000..7bd5a66 --- /dev/null +++ b/transcripts/ch1-welcome/3.txt @@ -0,0 +1,97 @@ +00:00 So let's talk about what we're going to cover in this course. +00:03 We're going to start by getting your machine set up and ready to go. +00:07 It's important that you follow along in this course, +00:09 that you do the code examples, that you play with the database, +00:11 that's how we learn as developers. +00:14 So the first thing we are going to do is walk through +00:16 how to set up your operating system with MongoDB +00:18 and the various other tools that we'll talk about later. +00:21 Whether you're using MacOS, Windows or Linux, +00:23 we have a video that shows you how to set this up correctly on your machine. +00:27 Next, we'll dig into what is NoSQL, why do we want to use it, +00:31 what are document databases, I touched a little bit on this in the beginning, +00:35 but we'll go deep into document databases, how they work, +00:37 and the trade-offs and benefits that we get from them. +00:41 We'll then start work with MongoDB proper, +00:43 we're going to fire up MongoDB +00:45 and we're going to connect to it with its native shell +00:47 and understand its low level native query syntax. +00:50 If this was a relational database, this would be kind of like studying SQL, +00:54 the TSQL language, but MongoDB doesn't use SQL, +00:58 it uses its own query language +01:00 which is easily understandable, but it is not the same. +01:03 Now this is actually going to be in Javascript for the most part; +01:06 you might be thinking this is a Python course, I don't want to learn Javascript. +01:09 Well, for the most part we're going to write our code in Python +01:13 and we're not going to do anything in Javascript, +01:15 but you need to understand how the query syntax +01:18 of the various tools and libraries in Python ultimately map down +01:22 to what you could work with in the management tools, +01:25 and in the management tools it's Javascript in the native query syntax. +01:29 We'll make sure that we cover that really well here as a great foundation. +01:32 Next up, we're going to talk about modeling data +01:36 with documents rather than tables. +01:40 You may have heard of third normal form +01:41 and modeling through normalizing data in a relational database +01:45 and to some degree, that knowledge carries forward into document databases, +01:49 but there are certainly many other trade-offs +01:52 and different types of modeling scenarios that you want to follow, +01:56 you certainly don't want to just normalize your document database +01:59 you'd be missing all of the benefits and getting some of the drawbacks potentially. +02:03 So we'll see that in document databases in general, +02:06 in MongoDB in particular, you model your data a little bit differently +02:10 and have different trade-offs and considerations, and we'll talk about that here. +02:14 At this point, it's time to start writing the code with Python. +02:16 So we will begin at the primary lowest level +02:19 that we can work with MongoDB, and this is PyMongo. +02:23 So here we're going to work in a query language +02:26 that is very, very similar to MongoDB's native Javascript language +02:31 but we're going to do it from Python, and this works great, +02:34 basically you're exchanging dictionaries and it's very fast and efficient. +02:38 However, sometimes it makes a lot more sense +02:40 to not just pass loosely typed dictionaries around +02:44 but rather rich classes with lots of functionality and structure. +02:48 So we're going to also talk about MongoEngine +02:51 which is an ODM, object document mapper for MongoDB; +02:56 think of this as an ORM, but because there is no relational bit, +02:59 it's not a relational database, we call it document. +03:02 So ODM for MongoDB and MongoEngine is one of the best ones, +03:07 it works really well in Python 3 and Python 2, +03:10 it has a whole bunch of features and different things +03:13 you can add to your application +03:15 on top of what MongoDB the database itself provides, +03:18 for example like type checking, things like that, +03:21 so really, really nice and you'll see that MongoEngine +03:23 is a great addition to what you might be doing. +03:26 Once we get the programming side in place +03:28 we want to take our database and add tons of data to it, +03:31 so we're going to take a simple example that we were playing with before +03:34 and add something that has effectively millions or at least million records in it +03:39 and then we're going to start interacting with it from MongoEngine, +03:42 just as well it could have been PyMongo, right, +03:45 we're going to start interacting with this database with lots of data +03:47 and see that it doesn't perform quite as well as we hoped, +03:50 maybe as well as all the hype around +03:53 MongoDB being fast would make you expect. +03:56 So we're going to see that we can take this server +03:59 and it's kind of ok on its own, if we left it alone +04:01 and we'll make it like five hundred times faster for some totally reasonable operations +04:05 so we're going to talk about the various knobs and tools +04:08 we have to make MongoDB really fast. +04:10 They're not hard to do, but they are not automatic +04:13 so you definitely want to learn about those. +04:15 And finally, we're going to take all of what we've learned +04:18 and deploy it into a cloud multi server environment +04:22 so we are going to create what would be a fake web app, +04:25 we'll just have a little Python script that stands in for the web app, +04:28 put that on one server up in a cloud computing environment, +04:30 on another one, we're going to set up a production hardened MongoDB server +04:35 and we're going to make sure that MongoDB production server +04:38 is totally locked down, running as safe as possible. +04:42 There are a lot of non obvious things about running MongoDB in production +04:46 and we want to make sure that it's working really well for us, +04:50 so we're going to go through this section and do five or six different things +04:53 to get MongoDB ready to be our production database. +04:57 And that's it, this is what we cover the course, +04:59 I think it's a really comprehensive introduction to MongoDB from Python, +05:04 and I really hope you enjoy it, I really enjoyed creating it for you. \ No newline at end of file diff --git a/transcripts/ch1-welcome/4.txt b/transcripts/ch1-welcome/4.txt new file mode 100644 index 0000000..d6a0f5d --- /dev/null +++ b/transcripts/ch1-welcome/4.txt @@ -0,0 +1,25 @@ +00:01 So let's talk a little bit about the tools that we're going to use. +00:03 We talked about Python, PyMongo and MongoEngine +00:06 as the programming language; +00:08 obviously we're going to use MongoDb as the database, +00:10 but we also learn a few other things during this course. +00:12 You'll learn about a management tool called Robomongo. +00:15 Robomongo is hands down the best way to work with MongoDB +00:20 from the client side, it gives you all the power +00:24 of the command line interface that comes with MongoDB, +00:26 but a great GUI kind of wrapped around it, +00:29 that's all I am going to say about it now, +00:31 but it's really fabulous and I think you'll enjoy. +00:33 Well see that we can even use Robomongo +00:35 to manage securely our production environment on a remote server. +00:39 We're going to be using PyCharm as the ide for editing all of our Python code, +00:45 so you'll learn a whole bunch of things about PyCharm, +00:47 if you don't want to use PyCharm, and you want to use +00:49 Sublime Text, or Emacs, or whatever, it's totally fine, +00:51 but we're going to be using PyCharm +00:54 and it's really great, I'll show you many of the tricks and techniques, +00:56 and there are just better ways of using it. +00:59 And finally, when we get to our deployment step +01:01 we're going to be working with Ubuntu, +01:03 and you'll learn how to set up MongoDB properly +01:06 in a production environment on Ubuntu. \ No newline at end of file diff --git a/transcripts/ch1-welcome/5.txt b/transcripts/ch1-welcome/5.txt new file mode 100644 index 0000000..7baa912 --- /dev/null +++ b/transcripts/ch1-welcome/5.txt @@ -0,0 +1,15 @@ +00:01 You might be wondering, hey who is this guy talking to me anyway? +00:03 Here I am, hi nice to meet you, my name is Michael, +00:05 you can find me on Twitter where I'm @mkennedy +00:08 and just a little bit of my background; +00:10 so I run the Talk Python To Me podcast +00:12 as well as I am the founder and I've written many of the courses +00:16 at Talk Python Training, for example this one. +00:19 And I'm even in the MongoDB Masters program +00:22 which is a group of about 30 people who are external advisers to MongoDB, +00:28 who work pretty closely with the teams there and give them feedback +00:33 on how MongoDB is working in the real world, in our environments. +00:36 So I hope to bundle up all of this experience and package it into this course, +00:41 and really give you some great take aways. +00:44 I'm so happy you joined my course, +00:46 I'm looking forward to teaching you a bunch of stuff— let's get started. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/1.txt b/transcripts/ch2-what-is-nosql/1.txt new file mode 100644 index 0000000..4e84613 --- /dev/null +++ b/transcripts/ch2-what-is-nosql/1.txt @@ -0,0 +1,48 @@ +00:01 Let's begin our exploration of MongoDB at a high level +00:04 with the why and history of NoSQL in general. +00:08 So if you look around the software development landscape +00:11 you'll see that people when they talk about +00:13 how are we going to design our application frequently say +00:15 well let's assume that we have a relational database, +00:18 now we can discuss what type of ORM should we use, +00:22 or should we use an ORM at all, +00:24 or should we use micro services or things like this. +00:26 Basically, the fact that we're starting from a relational database +00:29 is considered to be an axiom of software development, +00:33 we have data, it goes into relational database +00:35 now let's talk about the architecture, now let's talk about scaling, +00:38 let's talk about performance and so on. +00:40 And just to drive home how strong of a statement that is, +00:42 an axiom, recall the exact definition— +00:45 a statement or proposition that is regarded as being self evidently true, +00:49 it's just clear that you start from a relational database. +00:52 So what are the things I hope you take away from this course is +00:55 that the database style, the database engine is a choice, +01:00 it's a really important choice that has actually super important +01:04 and far reaching implication for your application. +01:06 So I want to sort of break this mold +01:09 that starting from a relational database is an axiom. +01:12 Sometimes it makes sense, sometimes it's perfect, +01:15 but a lot of times as you'll see throughout this course +01:17 starting with a document database is actually a better choice. +01:20 Now, what is NoSQL? Ask five people what NoSQL is, +01:24 you'll probably get five different answers back. +01:26 Some people will say, well the 'no' stands for 'not only' +01:31 so NoSQL is 'not only SQL', +01:34 well that's a great open minded view of the world, +01:37 but I'm sorry to say that's not what NoSQL is. +01:40 Maybe it means it doesn't have SQL, +01:44 maybe it means the system operates without the SQL language, right, +01:47 without select * from this etc, without that language. +01:51 If we look at the history, I think you'll see that this is also not the case. +01:55 Here's a toaster, this toaster operates without SQL, +01:58 is it a NoSQL toaster— I don't think so! +02:01 And of course not, it's not a NoSQL toaster, it's just a toaster. +02:05 NoSQL doesn't mean it operates without the SQL query language +02:08 in fact, I believe that document DB, Microsoft's document database +02:13 that runs in Azure, actually more or less uses +02:16 a flavor of the SQL query language to query it. +02:19 So no, it's not about excluding this SQL query language, +02:22 it's something entirely different, so let's next look at the history, +02:26 and I think you'll have a really good idea of what NoSQL is +02:29 and maybe it will come to a little bit closer agreement on the definition of NoSQL. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/2.txt b/transcripts/ch2-what-is-nosql/2.txt new file mode 100644 index 0000000..a6c0f4d --- /dev/null +++ b/transcripts/ch2-what-is-nosql/2.txt @@ -0,0 +1,38 @@ +00:01 The first record of what you might consider modern day NoSQL, +00:05 there were some older versions much, much older +00:08 about object databases that don't really carry on through today; +00:12 but what we think of when we talk about NoSQL today +00:15 really started back 2009 in San Francisco. +00:18 So this guy Johan Oskarsson, who at the time was working at last.fm +00:23 was getting together like a big data/ scaling databases +00:27 type of meetup in San Francisco, and the idea was +00:31 we're going to talk about open source databases, +00:34 distributed databases, that is databases that are easily horizontally scalable, +00:39 and that might not be traditionally relational. +00:42 This description here on the right actually comes from Wikipedia, +00:46 the name itself, the actual NoSQL, the word, +00:50 I don't believe it's here, but it was in a previous accounting, it's not in Wikipedia, +00:55 which if I could find the reference, I should go back and edit it, +00:59 is there's another guy named Eric Evans +01:02 who was attending this meeting as well +01:05 and Johan said hey what are we going to call this meeting +01:07 like we do have a name for these types of groups, +01:09 this type of thing that we're doing, and let's try to get something short, +01:12 like say a hashtag that we can use on Twitter to talk about it; +01:16 So Eric Evans said how about #NoSQL, right, +01:19 and that is the origin of the modern day term. +01:22 And the idea was, it was meant to describe this group of people +01:26 mostly running web apps with lots of data, +01:29 with high performance implications, or requirements, +01:32 getting together to talk about how can we give up +01:34 some of the features of relational databases to enable other types of things, +01:39 so maybe we'll give up atomicity, the acid properties, +01:43 maybe we'll give up joins, maybe we'll give up transactions, things like that, +01:48 and if we do that, how do we maybe structure our data differently, +01:51 how do we structure our databases differently, +01:54 to be better at basically being cluster friendly. +01:57 Alright, so to me, this is the idea of what a NoSQL database is, +02:02 it's a database that gives up some of the relational +02:05 database features or requirements or properties, +02:08 so that it is more cluster friendly, it is more friendly to scaling +02:13 and sharding and things like that. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/3.txt b/transcripts/ch2-what-is-nosql/3.txt new file mode 100644 index 0000000..2e3d73c --- /dev/null +++ b/transcripts/ch2-what-is-nosql/3.txt @@ -0,0 +1,80 @@ +00:01 So this historical description indicates that there are +00:03 a variety of different types of NoSQL databases that we've arrived at, +00:07 basically different levels or types of trade-offs that you might make +00:11 to encourage this horizontal scalable, cluster friendly type of data. +00:15 Now, if you go and look this up, what are the types of NoSQL systems +00:18 you'll see there are four distinct types, +00:21 but I think that would be four types under the not only SQL style systems +00:26 which as that my contention, that's not what NoSQL is. +00:30 So, I think there's actually three types of NoSQL databases +00:33 and then there's another thing that is not SQL; we'll talk about the four. +00:37 So at the most basic, basic level, the simplest ones are +00:41 I'm going to basically create a distributed dictionary +00:44 with id key that I can look up and some opaque blob that I can get back +00:49 so here is some thing, it could be text, it could be binary object, graph, +00:54 it doesn't really matter, store to this key +00:57 and if I ask for that key back give me the thing, +00:59 so for example, I want to store a user and all the stuff +01:03 that is associated with user into a single blob in Redis, +01:06 and when I ask for it back by user id, giving that thing back. +01:11 That means frequently these databases, these key value stores +01:14 don't let you query inside the opaque blob, even if it's text, +01:18 the idea is you query by the id, you get the thing, +01:21 but you can't easily ask it questions like +01:25 what is the average price of the order +01:28 of all the customers stored in your key value store. +01:31 Now, there are ways to add like additional metadata +01:34 that you can stick on to them, so you can sort of fake it a little bit on some of them +01:38 but in general, I can ask for the thing by a key +01:42 and I get a blob that's opaque to the database back. +01:45 Now, these are not great as databases in terms of general purpose databases +01:49 because you can't ask them a wide variety of interesting questions. +01:53 However, they are great in terms of this cluster ability +01:56 of the distributed cash type of thing there is super, super fast, +02:01 because every single query is just finally one item +02:04 by primary key and get it back to me +02:08 so if you horizontally scale that out, you can find +02:10 which key range maps to which server, +02:12 go to that server get it by id— boom, super, super fast. +02:15 So these are nice, they are often used for caches and things like this. +02:19 Next, we have what I think is the real sweet spot for NoSQL databases, +02:25 the ones that potentially could replace +02:27 traditional relational databases in your application, +02:30 that is they are flexible and powerful enough to be general purpose databases +02:35 and that's the document databases. +02:37 So this is MongoDB, CouchDB, OrientDB, +02:40 DocumentDB from Microsoft Azure, things like that, +02:43 so this is really where we going to focus +02:46 almost this entire class, so we'll come back to this. +02:49 We also have a more interesting type of database +02:52 called a columnar database, traditional relational database systems +02:56 like MySQL, Microsoft SQL server etc, store data in rows, +03:01 but a column oriented database is different +03:04 in that it stores data and its tables as columns instead of rows; +03:07 so it's in a lot of ways really similar to relational databases, +03:11 but you'll find that it's easier to kind of associate +03:14 what you might think of is like a pre-joint data, +03:16 maybe some orders, and maybe the orders have order items +03:19 and there's a bunch of those, so you might have one order +03:22 with multiple items, it's easier to group those together in a columnar database. +03:26 But they're kind of more or less like relational databases in a lot of ways. +03:30 So these I believe are the three types of NoSQL databases. +03:32 There's a fourth that often gets grouped here, +03:35 so let's talk about it— graph databases. +03:37 So graph databases let you model relationships +03:40 on many, many small interconnected things +03:44 so think of like social graphs, friends, the pictures, +03:48 the people who have shared that picture, you've liked that picture, +03:52 the friends of your friends who have liked that particular picture +03:55 you can traverse these relationships incredibly easy in graph databases +03:59 because you can actually query directly on the relationships. +04:03 Show all the things related in this way to this item and so on, +04:06 but that does not lead to this cluster friendly sort of thing, +04:09 in fact, this leads to being even more tightly connected +04:12 the less easy to map across multiple +04:15 horizontally scaled servers and things like that. +04:18 So in my mind, the graph databases are super interesting +04:21 but they're not NoSQL they're just not-SQL. +04:25 So that leaves us with three types— +04:27 key value stores, document databases and columnar databases. +04:30 So let's now continue on to talk about document databases. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/4.txt b/transcripts/ch2-what-is-nosql/4.txt new file mode 100644 index 0000000..be21aab --- /dev/null +++ b/transcripts/ch2-what-is-nosql/4.txt @@ -0,0 +1,85 @@ +00:01 So let's talk about how document databases work. +00:04 Here's a record from my actual online training platform +00:09 chapter 1001 from 'The Python Jumpstart by Building Ten Apps' course +00:15 and this is more or less exactly what came out of the database, +00:19 with a few things taken away so it actually fits on the slide here. +00:23 Now, let's break this into two pieces here, +00:26 this green piece and the blue piece. +00:29 First of all, you can see we have json, +00:32 when you work with document databases, you'll frequently run into json +00:35 as at least the visual representation of the record. +00:40 In fact, in MongoDB it doesn't really work in terms of json +00:46 it works in something called bson or binary json, +00:50 so this binary tokenised type typefull, rich typed version +00:57 of sort of extended json but already tokenised and stored as a binary version; +01:02 this is what's transferred on the wire and to some degree +01:07 this is what stored actually in the database +01:10 so how it actually get stored is it moves around +01:13 and the database storage engines are changing +01:16 and sort of plugable now in MongoDB, +01:19 but more or less you can think of this becoming a binary thing +01:22 and then stored in the database. +01:24 When it comes over into say Python, we'll of course map this into +01:28 something like let's say a Python dictionary +01:31 or a type that has these fields and so on. +01:35 So if you look at the green area, this is just the jasonified version +01:39 of any other database record, +01:41 it has let's think of it as columns if you will for a minute +01:46 it would have columns here like one would be the id +01:49 one would be the title, one might be course id and it has values; +01:51 and that's all well and good, until we get to lectures, +01:54 and here's where the power of document databases comes in, +01:57 lectures is not just like seven, there are seven lectures or whatever +02:01 no, lectures is a list, so multiple things, and each one of those things is +02:09 a lecture, an individual one, with its individual fields +02:13 so id, title, video url, duration in seconds, +02:16 again there's actually more to it, but so it fits on screen right; +02:18 with this document database, you can think of these things +02:22 as kind of pre-computed joints, and this solves a ton of problems +02:27 and makes the NoSQL aspect of document databases super powerful. +02:31 So it makes this chapter more self contained, if I want to get this chapter back, +02:35 instead of going to the chapter and then doing a join +02:39 against the lectures and maybe some other type of join, +02:42 and you're getting a bunch of different pieces and pulling them back together +02:45 I just might do a query, find me the chapter with id 1001 +02:50 bam, it's back, I've got the whole thing +02:53 and so you can think of this as like pre-joined data +02:57 if 80, 90 percent of the time I'm working with a chapter, +02:59 I care about the lecture data being there, +03:01 why not store it in a way that it's already bound together, +03:05 so I don't have to do that join, +03:07 I don't have to do multiple queries or things like this. +03:10 Okay, so this is really powerful and we'll talk a lot +03:14 about when this makes sense, when it does not make sense and so on, +03:18 but this means that if I take the single record +03:21 and I put it on some server, even if I've got like ten servers +03:25 and some sort of horizontal scale situation +03:27 and I do a query by chapter id, I don't then have to go back to the cluster +03:31 find where all the lecture data lives or anything like that. +03:34 No, it's just bringing that one record +03:36 brings most of the data that I need to work with +03:39 when I'm working with a chapter, right along with it, which is excellent. +03:43 That's the benefit, the important question the critical question to say +03:47 like is this going to work for our database system as a general thing +03:51 is well can I ask the questions that I would still have asked +03:54 if lectures was its separate table, if it was a flat table just like relational databases. +03:59 So, what if I want to find his last lecture here, 10 106, +04:06 will I be able to go to the database and say +04:08 hey document database, I would like to get lecture 10 106 +04:15 and I want to do that with an index +04:17 and I want to have it basically instantaneously, +04:19 even if there's a million records, I want to instantaneously get the record +04:23 that embedded deep down within it could be many, many levels +04:27 not just one, right, but in this case it's just one; +04:30 I want to get the record that deep down within it +04:32 somewhere matches the fact that the lecture id is 10 106. +04:37 And the answer is for the document databases yes, +04:41 so this makes them very, very different +04:44 than the key value stores just doing a json blob +04:46 because we can ask these very interesting questions, +04:49 we can do map reduce or aggregation type things for big data, +04:52 analysis and analytics, all sorts of stuff is possible, +04:56 because you can actually query deeply down into these nested objects. +05:01 So that's how document databases work, +05:04 and we'll explore the when, whys and hows of designing these documents +05:08 when we get to the document design chapter. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/5.txt b/transcripts/ch2-what-is-nosql/5.txt new file mode 100644 index 0000000..61ab7aa --- /dev/null +++ b/transcripts/ch2-what-is-nosql/5.txt @@ -0,0 +1,58 @@ +00:01 Now that we've talked about what NoSQL is, +00:03 where it came from and a little bit about document databases, +00:05 let's move to focusing specifically on MongoDB. +00:10 First off, I want to point out that MongoDB is open source +00:14 so if we come down here, you can see we've got +00:18 github.com/mongodb/mongo, there's actually all what are called drivers +00:23 so like how do you access MongoDB from Python, or CSharp, or Java or whatever, +00:28 and a bunch of other stuff out here on their github, +00:31 but Mongo is the actual database server. +00:33 So it's cool that it's open source, now there's a lot of things that are on github +00:38 that are technically open source, but not really active, +00:41 it's like oh that's been changed, four years ago, +00:45 and it's got 50 pull requests that haven't been +00:47 even addressed in the last six months. +00:50 That's not good; that's not the case with Mongo, obviously, +00:53 eleven thousand stars, three thousand forks, +00:55 when was the last check in— it was three hours ago, okay? +00:59 So that's really awesome, and they're fixing the build on 'Windos' +01:05 so that must be a slightly different version of Windows, +01:08 I'm just kidding, I'm sure they lost the w there +01:10 but you know, it runs on the major platforms, +01:12 Windows, Linux and MacOS, if we look over at the pull request +01:17 you can see these are only a few days old, +01:20 there's 1129 that are closed, so these are all really good signs +01:25 for MongoDB's open source site, +01:28 open source repository to be active and real, +01:31 not just it's up here, but it's a really active thing +01:35 with a huge company and hundreds of engineers working on it. +01:39 If we look at what it's made of, it's basically a C++ app, +01:43 there is a decent amount of Javascript +01:45 and there as well as we'll see Javascript is fundamental +01:48 in the raw query api and some other stuff. +01:50 Now we can get the source code here, but that's not how we get MongoDB. +01:54 So the way we get MongoDB is we go to mongodb.com, +01:57 and we click download and it takes us here, +02:00 so you can see there is the free community server version +02:03 and if we come over here we could get the OS10 version with ssl +02:07 we could download it just as a binary +02:10 or we could install it with home brew, that's pretty cool. +02:13 If we want to install it on Linux, let's pick something, +02:16 let's say we want to install it on Ubuntu, +02:19 okay, here is how you install it with aptitude, right +02:22 so on Windows you just get an msi and install it directly that way. +02:26 But notice, they all have ssl support, I believe there's a time +02:30 when ssl support was actually an enterprise feature or like a paid feature, +02:34 but thankfully that went away, and the community edition has this right here. +02:39 So if you get a chance, I would recommend installing it +02:42 especially on Linux from one of the package manager type things +02:46 like here with apt, because then the underlying system will know +02:50 that there's an update for your database server, +02:54 not a feature on Windows, you just have to keep track. +02:57 You can go and get the enterprise server which has an in memory version, +03:00 an encrypted at rest version, and sort of +03:04 advanced identity control features within it +03:10 but if you just want a standard database without those things +03:13 you don't care about say encryption at rest +03:16 or integration with active directory or something like that, +03:18 then the free version is totally good. \ No newline at end of file diff --git a/transcripts/ch2-what-is-nosql/6.txt b/transcripts/ch2-what-is-nosql/6.txt new file mode 100644 index 0000000..4ee61be --- /dev/null +++ b/transcripts/ch2-what-is-nosql/6.txt @@ -0,0 +1,106 @@ +00:01 Now let's look a little bit at who is using MongoDB and how. +00:04 On one hand, it's not that important that it's a popularity contest— +00:07 does it solve your problem, good, use it. +00:09 On the other, MongoDB is different, right, +00:12 it's not a relational database that people have been using for thirty years +00:15 and we call that axiom conversation, I had at the beginning, +00:18 if you are the one adopting MongoDB you have to take this idea +00:21 and present it to the people that run the business, +00:24 to your managers, to the tech team and say +00:27 hey this is a safe thing for us to do, this is a good thing for us to do. +00:30 And so, by looking at the other users of MongoDB, +00:34 how they're using it and how much data and traffic they are passing through it +00:37 can give you some really good support +00:41 like hey look it's working well for these companies, +00:43 and they're way more risk adverse than we are, +00:45 so if they can use it, we can totally use it. +00:47 So, with that in mind, let's go look at who uses MongoDB, +00:51 so they have a whole page who uses MongoDB right here, +00:54 we can flip through and there's a few major ones; +00:56 so we've got MetLife, they're doing some pretty interesting things +01:00 obviously they are a large insurance company +01:02 they have a single view of a hundred million customers +01:05 across 70 systems and they built this whole thing up on Mongo +01:07 and it's 90 days, that's pretty cool. +01:10 Expedia uses it for millions of customers +01:13 while they're looking for travel, that's great. +01:15 Now let's look at some more, you can see the scrollbar, this is actually huge, +01:19 so let's scroll down to find some interesting ones. +01:22 So let's say Royal Bank of Scotland, +01:24 this supports the bank's enterprise data services +01:26 underpinning several core trading systems, +01:29 okay that's intense, right, like if you're debating +01:32 whether or not this can do like you know +01:34 some probably not super intense +01:36 for the majority of the students part of your app, +01:38 if Royal Bank of Scotland is going to make this part of their core trading systems +01:42 that's really putting a lot of faith in it. +01:46 Biotech, they use this to accelerate their drug testing, +01:50 Facebook, they have a whole bunch of interesting things +01:52 that they're doing with Mongo, they ran like a backend as a service of Mongo +01:57 when they acquired Parse, but they're not doing platform stuff like they used to. +02:00 Now let's flip around, let's have a look at say ebay +02:03 they're doing delivering all their media metadata with five nines for liability; +02:07 Barclays, a big bank, so they've replaced +02:10 a whole bunch of relational systems there, +02:13 let's keep going, come down here to our friends in Germany, +02:15 they built a pretty amazing internet of things platform on top of MongoDB, +02:20 come down look the New York Times, +02:23 they basically did all their social sharing activity on top of MongoDB, +02:27 Business Insider, you probably run across Business Insider the website, +02:31 so they've been around since 2009, +02:34 they launched in New York city, and their whole site +02:37 runs on MongoDB, which is pretty awesome. +02:39 Speaking of business, let's look at Forbes, +02:43 they rebuilt their whole cms on top of MongoDb, +02:45 resulting in a jump of 5 to 15 percent in mobile traffic overnight, that's really cool. +02:50 So Carfax, they sell cars online and in person +02:54 so a ton of traffic happening there, that's really cool. +02:57 Cern, I love Cern, these guys at the Large Hadron Colider +03:02 they're using MongoDB to manage the data +03:06 while they're searching for the Higgs Boson +03:10 which I think this probably needs updating +03:12 because as they now have found the Higgs Boson +03:14 and won the Nobel Prize as a part of that. +03:16 Another interesting a long time user of MongoDB is Foursquare; +03:21 so Foursquare is as far as I know more or less entirely powered by MongoDB +03:26 and here you can say it powers the processing storage of all check ins +03:30 with hundreds of thousands of IOPS on MongoDB, +03:34 that's hundreds of thousands of operations, input/ output operations +03:37 per second on MongoDB, which is really, really cool. +03:40 Let's look at Sailthru, so Sailthru is like marketing email campaign company +03:46 and they store 40 terabytes of data in MongoDB across a 120 nodes +03:53 so remember we talked about document databases +03:57 and NoSQL databases in general being good for horizontal scale +04:01 and sharding and partitioning your data; +04:04 120 nodes in your cluster that's pretty intense. +04:07 All right, let's do one more, let's talk about Shutterfly, +04:10 so Shutterfly is like a photo sharing site, pretty cool, +04:15 you can like put your pictures there, sharing with people +04:18 you can get like printed books they were doing that before +04:20 some of the main companies like google were and so on, +04:22 so this is interesting in that they have a bunch of projects, +04:26 on Mongo storing over 20 terabytes of data. +04:29 Square Space, Stripe and on and on it goes, right, +04:33 all of these really cool companies are using MongoDB. +04:37 I guess one more let's look at UnderArmor here. +04:40 So Under Armour is interesting because I haven't seen +04:43 any of the previous examples explicitly calling this out; +04:46 so Under Armor is like an athletic clothing company in the US +04:49 and around the world, and their online shop is powered by MongoDB +04:54 and it does over two billion dollars in sales, so that's pretty awesome. +04:59 All right, so why do we spend all this time talking about who uses MongoDB? +05:02 One, to show you there are a bunch of companies +05:05 being really, really successful with MongoDB +05:07 and that there are different use cases, +05:10 different companies in different areas doing different things, +05:13 we saw like biotech, we saw pharma, here is e-commerce, all sorts of things. +05:18 Oh, EA, I didn't pull up EA but they're using it to power, let's go up here to EA, +05:23 so EA is using this to scale their online games to millions of players; +05:29 so all sorts of really cool and interesting use cases +05:33 that you can use to say hey, we should give this database a try +05:37 because here's a bunch of other people being successful with it. +05:40 This also means that you can rely on Mongo +05:43 because it's taken a serious beating from quite a few different angles +05:47 and use cases, it's not some barely used database +05:50 but it's highly, highly used actually. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/1.txt b/transcripts/ch3-computer-setup/1.txt new file mode 100644 index 0000000..7acf92f --- /dev/null +++ b/transcripts/ch3-computer-setup/1.txt @@ -0,0 +1,89 @@ +00:01 In order for you to get the most out of this course +00:03 you're going to need to fallow along. +00:05 We were talking about the Mongo shell you should open it up and play around. +00:07 When we work with PyMongo, you should pip install it +00:10 and write some code to talk to your local MongoDB server. +00:13 When we're doing MongoEngine or working with indexes, +00:15 again, you should follow along and do these things. +00:18 In order to do that, you're going to need some software, +00:20 you're going to need some starter code to get going +00:22 and you're going to need basically to have MongoDB set up and configured correctly. +00:26 So in this part of the course, let's talk about +00:28 getting your machine set up so you can follow along. +00:30 This is a course about MongoDB, so it shouldn't be +00:33 terribly surprising that it's going to require MongoDB. +00:35 Now if you look across the bottom here you can see +00:37 there is a version for Windows, Linux and MacOS. +00:40 So regardless of what operating system you are using +00:43 you should be able to use MongoDB installed locally and work with it there. +00:48 There are hosted services, places like ObjectRocket and Mlab +00:52 and if for some reason you can't install MongoDB and configure it, +00:55 unlikely, but possible, you could actually connect to one of those services. +00:59 But we're going to assume that you can set it up locally +01:01 and I will walk you through step by step +01:03 how to do that for each and every operating system below, +01:06 with the exclusion of Solaris of course. +01:08 Now, this is MongoDB for Python Developers +01:11 so it shouldn't surprise you that hey we're going to need Python, +01:13 and we're focusing on Python 3, so most new projects are created in Python 3, +01:17 it's the future of Python, so we're definitely focusing on Python 3, +01:21 that said, the things we're doing are not super specific to Python 3, +01:24 it should pretty much work across all the versions +01:27 in case you happen to be using a legacy Python. +01:29 So do you need to install Python 3? +01:31 Well that depends if you're working on Ubuntu, +01:34 you probably already have at least Python 3.5 on your system. +01:37 If you're on MacOS, by default you have Python, +01:40 but only legacy Python, only Python 2, not Python 3, so you'll need to install that. +01:44 And if you are on Windows, unless you've done something special +01:48 there is no version of Python, so make sure +01:50 you get Python 3 from python.org, download and install it. +01:54 Now we're going to write a lot of code in here, that's good, +01:56 I think that's the way coding course should be, +01:59 and we're going to use the editor from Jetbrains called PyCharm. +02:03 In my opinion, this is the best tool for working with Python code +02:07 and you'll even have plugins for MongoDB if you go and search +02:10 their tool repository, so we're going to use PyCharm. +02:13 Now, PyCharm is available in two flavors, +02:16 there is a community free open source edition, and there's the pro edition. +02:21 If you have the pro edition, feel free to use that, +02:23 but if you don't, you can grab the community edition, +02:26 it will do everything we need for this course. +02:29 If you want to use some other editor, that's totally fine, +02:32 you can use whatever you like, but if you want to follow along exactly, +02:35 I recommend you give PyCharm a shot. +02:37 There is a couple of ways we can work with MongoDB once we have it installed, +02:40 we can use the cli the command line interface to it that comes with MongoDb itself, +02:46 or we can use something called RoboMongo. +02:49 So RoboMongo in my opinion is the best way to work with MongoDB +02:54 the idea is you can see a little dark gray area, that's basically the shell +02:58 and you can type as if that was a command line interface. +03:01 However, it operates inside this gui so you could write a little bit of cli stuff +03:05 and then go interact with the stuff visually, and this is a really, really nice balance +03:09 of giving you the full power of MongoDB, but also a lot of visual support. +03:13 I think it's super productive and is great. +03:16 You can see there's screenshots for all the three major operating systems, +03:19 so whatever operating system you use, RoboMongo is going to work great, +03:23 it's also free and it's also open source, how about that. +03:26 Finally, when we write that code with PyCharm, +03:28 you're going to want to be able to take it with you. +03:31 Sometimes you might want to grab the finish code +03:33 that you saw me create in the video and run it, +03:35 other times, we might have started not from a blank empty file +03:38 but from some sort of starter code +03:41 that got us further along in the demo from the beginning. +03:43 We also have a couple of large databases +03:46 that you want to get access to for the performance section of the course, +03:50 all those and more are contained in this github repository here, +03:53 so github.com/mikeyckennedy/mongodb-for-python-developers +03:58 so be sure right now to pause this video, go over here +04:02 and star and maybe even fork this repository so you're sure to have it with you. +04:06 And also download or clone it to your local drive, +04:10 because you're going to want to have this to work from, as you go through the course. +04:13 So there you have it, that's the software source code +04:16 and tools that were going to use. What we're going to do next, +04:18 I'm going to walk you through each operating system, Windows, MacOS, and Linux +04:21 and show you how to set up the tools and how to configure MongoDB +04:25 and get everything working just right. +04:28 If you're a Linux person, there is no reason to watch the say Windows version, +04:32 so pick the video that matches your operating system, skip the others. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/2.txt b/transcripts/ch3-computer-setup/2.txt new file mode 100644 index 0000000..7951c97 --- /dev/null +++ b/transcripts/ch3-computer-setup/2.txt @@ -0,0 +1,167 @@ +00:01 Here we are in windows 10, so I think this might be the anniversary update +00:05 or not sure exactly which version, but it's not the fresh one, +00:08 it's been out for quite a while now. +00:10 So what we're going to do is we're going to install MongoDB. +00:13 Let's go over here, check out mongodb.com, click on download, +00:17 so we're going to go and get the msi, I want to get the one with ssl x64 bit, +00:23 you put your name in here to get info from MongoDB if you want, +00:26 you don't have to; okay downloaded, let's run this, all right, +00:32 so current version of MongoDB at the time of this recording is 3.4.4 +00:36 so we're going to install this, and I'll just open up the custom +00:39 complete would be totally fine, just so you can see what's there, +00:43 there's the server which is mongod itself, there's the client tools, +00:47 there is the monitoring like analysis tools, +00:50 import, export, for data backup and restore +00:53 the sharding server Mongo s and then like a few other utilities, +00:58 unless you have a good reason to not have these on your machine, +01:01 I would just go for the complete install. +01:06 All right, it looks like MongoDB is installed let's try this— +01:09 no Mongo, all right, the reason there's no Mongo is +01:14 we've got to set up this thing in our path, +01:17 so let's go over here to program files, mongodb, server, number, bin, +01:21 so basically to do anything interesting from the command line +01:25 and that's really where you want to be working with MongoDB +01:28 you're going to have to come over here +01:30 and put this into your path, so let's do that now. +01:33 You just go to properties, advanced, properties, environment variables, +01:37 this is way deep down, go to the path and hit edit, and then hit new, +01:41 this is very much nicer than the way it has been in the past, +01:44 and it will just take that path and put it here; +01:47 close everything off, ok, so now we should be able to run Mongo, +01:54 hey look it's not going to work, but we can see +01:58 it's this one that we just found and set up, +02:01 so in order for Mongo to work, we can actually try to run Mongod +02:05 and we're going to get a sad, sad message here, +02:09 so if you look somewhere it's going to say this directory +02:13 basically the database directory is not set up. +02:16 Well, there aren't default places where you can put the data +02:18 and it will actually create that, you see here is the startup settings that it's using. +02:23 So we don't want to do this, we want to actually make another one +02:28 logs and one called configs, so you get to configure all of these, +02:34 so you can configure that however you like, +02:37 but we should set up something like this +02:40 and so let's go in here, now I'm going to copy a config file over +02:44 so we have two, and notice I've named one command line and one is service, +02:49 let's just look at the command line one. +02:52 So notice, there's not a lot going on here, +02:54 I think this directoryPerDB we could actually drop this, +02:57 this is not used in the new version, +03:00 so we're basically saying drive c:\mongodb\data, +03:02 let's just double check that that does exist, it looks good up here, +03:06 c:\mongodb\data, okay, journaling enabled, you definitely want that on +03:11 and this is super important, listen on a local host only, only, +03:15 this is a dev machine there's no reason they should listen on the open internet +03:21 in case your firewall is down or you're somewhere +03:23 where people are scanning the local ports on their local network, +03:26 think hotel, something like that, so we don't want any of that to happen, +03:30 so we're going to listen on a local host only. +03:32 All right, so what we need to do now is +03:34 we want to try to run MongoDB again, now with this, +03:37 so let me go up here and copy the path, +03:43 so we should be able to run MongoDB now, let's clear that off, +03:46 so mongod and when we tried to run it before and it gave us this error, +03:50 now we can say --config is that, and if we've got everything set up correctly +03:55 this should work, there might be permissions on that mongo folder I created +03:58 we're going to find out in a second. +04:01 It looks like things are going well, let's go over here and try to connect +04:06 so we can type mongo and hey, we're in, +04:09 I can do things like show dbs what's here, perfect, +04:13 ok so it looks like this is working, it says now warning, +04:16 you don't have access control like this is wide open to the internet +04:20 and it's unrestricted read/ write, this is not the best, +04:24 it's pretty much okay because we're listening it on the local host, +04:27 still could be a problem, you might want to set up an account +04:30 when we get to the deployment and production settings, +04:33 this is, we're going to solve these problems, +04:35 but for development this is probably good. +04:37 I had that one cofig, this one that worked, +04:41 let's check this one out and make sure everything is ok as well. +04:43 So this service one is going to run +04:45 when we install MongoDB as a Windows service +04:48 so if we were running in like Windows virtual machine +04:52 in aws, ec2 or in Azure something like that, +04:56 this would be what we'd probably run, of course +05:00 with credentials and things like that, we'll talk about it at the end; +05:03 but if we're going to set this as a Windows service, +05:06 this will only succeed if we set the logs, +05:09 so that's why we created this logs folder +05:12 and that's why this service one has a system log section. +05:16 So the next thing to do, now that we're over here +05:19 is we actually want to first let's just test that, +05:23 so let's test this service version and we won't see anything because the log file +05:31 but if it just sets there, I guess we could go ahead +05:35 and test that we can connect to it— yeah, looks like that worked. +05:38 Okay so it looks like the service is working +05:43 we'll just control c out of there. +05:46 Now the next thing that we need to do, this is optional, +05:49 you don't have to do this, you could literally come and type this out every time, +05:53 but let's go ahead and set this up as a Windows service, +05:56 so you can tell it to auto start, delay start +05:58 or just flip open to the services and click go +06:01 whenever you want to use MongoDB, that's how I +06:03 whenever I'm working on windows, how I use it. +06:05 So we can go to the services, +06:11 and let's hit m to see if there is anything for Mongo, +06:13 and now there's nothing for MongoDB here, ok, so no MongoDB; +06:16 and what we want to do is we want to register MongoDB as a Windows service, +06:21 now there's something that's really, really important here, +06:24 I can run MongoDB like this, +06:28 -port equals whatever port,  --ssl and whatever, +06:35 all of the options go here, so --db path equals, we get filled this out here, +06:43 it turns out the way that MongoDB registers itself +06:47 if I try to install it as a Windows service using the explicitly passing the parameters +06:52 the only way to change those values, to change the way MongoDB works, +06:55 is to actually go and edit the registry entry in Windows, not amazing. +07:01 So what we're going to do instead, is we are going to do what we already did +07:04 we want to go to basically say run that config file. +07:07 Now, the other thing that I've seen can be really tricky +07:10 is the Windows service path might not be the same as your path +07:13 so you need to use like full path names everywhere, +07:16 so we'll say where mongod, so we want to run this explicitly +07:21 because that's what gets recorded in the Windows service, +07:24 so we're going to say that instead of just mongod, +07:27 we'll say --config and that was in c:\mongo\config\ this one, +07:36 now we've got to use the service one that has the log +07:38 and then finally, here's the trick, this is the thing, +07:42 actually this is not going to work, so I'm going to copy it, +07:45 I'll show you this not going to work. +07:47 So the trick is to say I would like to install this as a service +07:50 because it's not going to work, i'm going to copy it, so I don't have to type it again, +07:55 ready— enter, now, no errors, but if I refresh, also no MongoDB. +08:02 What happened? Well if you actually open up that log file +08:06 in there it will say permission denied, could not install MongoDB, +08:10 why— because this is not an administrator command prompt, +08:14 not root or anything like that, this is just straight up, just whatever my account is, +08:20 so I got to right click, you see these options, if you shift right click +08:24 you say run as administrator, and then you run the exact same command +08:28 and it does the exact same thing, +08:32 except in the log file, there's now a different message +08:34 if I refresh— ta-da, we have MongoDB. +08:38 So let's test this, if type mongo, trying to connect, trying to connect, +08:42 it's going to time out, right, nothing. +08:45 Now if I go over here and I press start, do the same thing again, +08:51 ta-da, now we have MongoDB set up as an auto start windows service. +08:55 That's pretty awesome right. +08:58 So if we reboot, MongoDB will run. +09:01 It might be the case that just sometimes you want to run Mongo +09:03 and the other times you don't want to it say sucking down the battery on your laptop, +09:06 you can set it to automatic delayed start, so your Windows boots faster, +09:10 and you'll still have it, or you can just set up purely to manual +09:13 in which case it's only going to run +09:15 after reboot if you come over here and click go. +09:19 So that's depending on how you use Mongo +09:23 whether you want it certainly in production +09:25 if you're on a Windows server set that to start +09:27 but maybe manual for development, I don't know, +09:30 it depends how often you use MongoDB, if you use it all time +09:32 that's all you work on is apps to talk to it, +09:34 just set it to auto start or delayed or something. +09:36 Okay, so now this Windows machine is all configured to run MongoDB, +09:41 how about PyCharm, and RobMongo and so on? +09:44 Those all have straightforward installers +09:47 so just install Python 3, Robomongo and PyCharm whichever edition you want +09:51 and just next, next, next your way through, +09:54 with the Python one make sure that you check the box +09:56 that says add Python to my path, that one is important. +09:59 Other than that, there's really nothing else to it, you'll have a machine +10:02 that is set up and ready to do this MongoDB course on Windows. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/3.txt b/transcripts/ch3-computer-setup/3.txt new file mode 100644 index 0000000..809fe43 --- /dev/null +++ b/transcripts/ch3-computer-setup/3.txt @@ -0,0 +1,45 @@ +00:01 Let's review installing MongoDB on Windows. +00:03 It just took a couple of steps, we started out +00:05 by downloading and running the msi installer, +00:08 and then, we realized well, we don't actually have access to Mongo or Mongod +00:13 or any of the other tools from our command line, so what are we going to do— +00:17 well it turns out that it got installed into this location +00:20 and notice that the number may vary based on the released version of MongoDB, +00:24 so we went in there and we actually added that to our path; +00:27 and then we could type mongo, mongod, mongo restore +00:31 and all the various tools are going to have to use throughout the course. +00:34 And then we said all right, well this is not going to run on its own, +00:37 the way it gets installed on windows is it assumes +00:40 that there is a c data mongo or data-db, +00:43 something like that, it'll tell you on the error, +00:45 but I don't like to put stuff and just see data, +00:47 I kind of want to group stuff a little bit more +00:49 so we made some mongodb folders, +00:51 we made the c:\mongodb\data,  \logs and \configs; +00:56 so those three obviously longs go in logs, data goes in data, +00:59 and then we have those two config files that we can use to run mongodb +01:03 with all the various settings set for us. +01:06 We copied the configs over, and I'll include those in the source controle +01:11 so you guys can grab the windows setup configs +01:14 and you know, just change the path to wherever you put things; +01:17 and then you want to test those configs, so we're going to test them +01:20 by saying mongod--config and the full path to the config, +01:24 this command one is meant to have no log in, so basically it spits out +01:29 the log information to the command line to the command shell, +01:32 that way you can see what's happening. +01:34 Of course, in the service version, the service actually won't install +01:37 if there's not somewhere for the logs to go to +01:39 because it knows you're not going to see anything so it has to log it for it to work. +01:42 All right, so this is just if you want to run it on the command line, +01:45 then we're going to install it as a windows service, +01:48 so this time we use the mongo-service config, +01:51 which is basically the same, other than having a log in +01:53 and we added the --install, okay. +01:57 And it was really important that we run that command prompt +02:01 as an administrator not as a regular user +02:03 otherwise it doesn't have access to alter the service configuration in windows. +02:08 All right, once this was all done, we went to the service config, +02:11 we pressed start and then we're good to go. +02:14 Final thing you might want to do, adjust the startup mode to manual, +02:18 depending on whether you want mongodb to start every time you turn on windows +02:22 or only when you click the button in the service panel. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/4.txt b/transcripts/ch3-computer-setup/4.txt new file mode 100644 index 0000000..d0ac67f --- /dev/null +++ b/transcripts/ch3-computer-setup/4.txt @@ -0,0 +1,69 @@ +00:01 Here we are on a relatively new Mac, you can see that this is MacOS  seirra, +00:06 so relatively new, and if I come over here and I type Mongo, +00:10 no there is no MongoDB installed here. +00:13 So what are we going to do— we're going to install and set up MongoDB, +00:17 so those of you who use MacOS, like I do, you'll see that this is actually +00:21 the easiest of all the various operating systems to set up. Let's do it. +00:25 So we're actually going to start out at Homebrew, +00:30 now you may have Homebrew installed which is awesome, +00:32 and you can just run brew update, but if you don't +00:34 then you're going to need to run this command; +00:37 so we're going to run a script pass it off the ruby, +00:40 this is going to install Homebrew and if I hit enter, +00:43 it's going to go do this, this one time it needs my password +00:46 to make changes to the system, but in general, +00:51 you should not sudo run Homebrew, it even warns you if you do that I believe. +00:56 +01:00 Okay, I get the little ding, it looks like everything is all set up. +01:04 So now if I type brew, yeey, stuff happens. +01:07 So the next thing that I want to do is actually install MongoDB, +01:10 so brew install mongodb, +01:14 +01:21 and just like that, after 15 seconds, 20 seconds something to this effect, +01:25 it says MongoDB is all set up. +01:28 Now before I clear this and just run it, notice there's a couple of things, +01:31 it tells us right away, right here how we get started, +01:34 we can either run MongoDB as a service, +01:36 and if I do this without sudo, it's going to run this as a service any time I log in, +01:42 if I do it with sudo, it's going to basically do this +01:44 as part of the machine startup, shut down, +01:47 or I can just run MongoDB like so. +01:49 So let's go ahead and set this as a service, +01:52 it takes a moment and now it's up and running. +01:58 So how do we know whether it's running? +02:01 Well first of all, if I type mongo, it has something there, it has an app there right +02:04 you can see 3.4.4 is the one at the time of this recording, +02:08 and now it's connected and there's a few warnings here about access control, +02:11 this is worth considering, if this was a production machine +02:16 I would be quite concerned about this, it's my developer machine, so I'm not. +02:22 Let me show you why I'm not. +02:24 Okay, so if we exit out of here, the other way that we could run MongoDB +02:28 it's already running, so this isn't going to work again, +02:31 but we could run it passing this config file, +02:33 but what's interesting is, check out this config file +02:36 so if we go look at that, it has just a few things for us, +02:39 it tells us where the log file is going, good for system services, +02:43 where the data is going, and most importantly, +02:48 it's listening only one local host, 127.0.0.1. +02:53 I don't know what my public ip address is or I have both the ipv6 and ipv4, +02:57 MongoDB is not listening to either of them, +03:00 moreover, I also have my firewall turned on as much as possible, +03:04 believe it or not, it is not turned on on MacOS by default +03:08 that is super, super suspicious to me. +03:10 But anyway, firewall's on, and we're not even listening on the public network interface. +03:15 So do not change that, make sure that you are listening on only local host +03:20 or that authentication warning, that's going to have a whole different meaning. +03:23 When we get to the actual deployment section, where we deploy to production +03:27 we're going to set up users, we're going to set up ssl, +03:29 we're going to set up authentication, all of those kinds of things, among others, +03:32 but for now, for the devmachine, I think listening on local host +03:36 is probably sufficient for what we're doing. +03:39 So this MacOS is ready to take the class. +03:43 The other things we would need to do, is install PyCharm, install RoboMongo, +03:47 and do we have Python 3— we do, we have Python 3 installed, +03:53 I believe I actually already installed that earlier on this virtual machine. +03:58 So you're going to want to make sure you have Python 3 installed, +04:00 RoboMongo, and PyCharm, they all have super simple installers +04:03 that's basically either drag and drop or double click +04:06 and follow the next button and you'll be all done. +04:08 Then the machine will be completely ready to take this class. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/5.txt b/transcripts/ch3-computer-setup/5.txt new file mode 100644 index 0000000..c436baa --- /dev/null +++ b/transcripts/ch3-computer-setup/5.txt @@ -0,0 +1,24 @@ +00:01 We're going to use homebrew, if you don't like homebrew +00:04 you can actually download a tarball and unzip it and set up all the stuff +00:07 but homebrew is much better on a couple of levels, +00:11 so make sure you have homebrew and that you update it, +00:14 and then to install, we're just going to install homebrew +00:17 and then we just brew install MongoDB, +00:19 wait about 20 seconds for it to do its magic, +00:22 we're going to start up MongoDB, there's two ways to do that +00:24 we could say brew services start monogodb +00:27 and that will actually register it as a system service +00:30 that starts every time you log in; +00:32 if you don't want to do that, you don't want to start it this way, it's fine +00:34 you can say mongodb--config and point at the config file. +00:38 If you want to make any changes, well, there's the config file +00:40 you can just edit that with whatever editor you like, +00:43 and you can change the security, you can change the ports, +00:46 whatever you want to change about MongoDB +00:48 just change this file, and stop and then start the service, +00:52 or just cancel out running this mongod command, and then run it again. +00:56 Final thing is, if you brew service start mongodb +00:59 and it's going to continue to autostart, if for some reason you want it to stop, +01:02 I believe it makes a file, a plist file, and it puts it in your home directory +01:07 for your user account in /library/launch/agents/ something involving mongodb, +01:13 so you can just delete that file I believe and that will stop it. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/6.txt b/transcripts/ch3-computer-setup/6.txt new file mode 100644 index 0000000..02f1678 --- /dev/null +++ b/transcripts/ch3-computer-setup/6.txt @@ -0,0 +1,82 @@ +00:01 Are you taking this class using your Linux desktop? +00:03 Well, let's get your machine all set up and configured with MongoDb. +00:07 So here we are just on the homepage mongodb.com, +00:10 I am going to go click download, and it's totally possible to click and say +00:16 download the tarball, but we could also say I would like to see +00:19 how to set this up on let's say Ubuntu, as you change this, +00:23 notice different things happen, so if we go to Amazon +00:27 it says here are instructions for installing with yum, that's the package manager there, +00:31 if I go to Ubuntu, so here's the instructions for installing with aptitude, +00:34 so we're going to go over here to that one, +00:36 make sure you pick the right distribution, do we want to read this— no. +00:40 So there's a couple of steps that we need to go through +00:43 and they're pretty simple, most importantly they just like walk us through it +00:47 so notice here that there is a package named mongodb.org +00:52 let's try to just install that, sudo apt install that, +01:00 oh it needs my password, okay, and nope, there's no mongodb, +01:05 darn, doesn't it say right here, here's how you install all the pieces? +01:08 It is true, but we got to go through a few steps to get there. +01:11 So first thing that we got to do is basically add a source to aptitude, +01:16 so we're going to go over here, and we're going to set a key, +01:19 so you're just going to go through a few copy paste steps, +01:21 so we're going to do our apt key here, it takes a moment, +01:26 and all right, that's all good, next thing to do +01:28 is we're going to create a list file, all right, great. +01:34 Be really careful about the version numbers here, +01:38 later is probably better, pick the one that matches yours. +01:40 So the next thing we need to do, is run a sudo apt update +01:44 and you can do apt.get or just apt, whatever +01:47 but you need to tell it hey, go pull all the sources +01:50 now the new one included and just have that list there. +01:53 We don't need to back up anything, so go away. +01:58 Alright, now everything is registered, we're pointing at the right package source, +02:02 we've updated it, now we can go over here and do +02:05 our thing that we tried to do originally. +02:08 So we wanted a sudo apt installed mongodb.org, +02:14 this time it knows what that means, hey look that's mongodb, +02:18 mongos which is a sharding server, mongo server, mongo shell, mongo tools, +02:22 I am just going to install them all. +02:25 +02:29 Perfect, okay, so we have MongoDB all set up and ready to go, +02:35 and now we can just type mongo, and it tries to connect, +02:40 we have mongo now but we really need to start it. +02:44 +02:51 So we started up mongod, great, now we can connect to it. +02:56 Awesome so it has some warnings again about access control +02:59 when we get to the deployment chapter, +03:02 we're actually going to set up Ubuntu as a cloud server +03:05 with some of these errors removed, +03:07 we're going to set it up with access control, with authentication, +03:10 with firewalls, all sorts of things, but for now, for the dev version, +03:13 we're going to just use this, okay. +03:16 So it looks like it's up and running, that's cool, +03:19 now the last thing is maybe we want to configure our server; +03:22 so, we can come over here to /etc/mongod/conf +03:30 and you can see we've got our storage path, +03:33 like here's where our data files are going to go, change that if you like, +03:36 journaling, you generally want that on, +03:38 it's going to be running with wired tiger, +03:41 it is an older style of database storage engine called this mmapv1, +03:45 that's how things used to work, they've switched the default to wired tiger +03:49 because it's faster, I believe it's much faster for inserts a little faster for reads, +03:53 here's where the log file goes, if it's not working and you want to see what's going on. +03:58 So most importantly though is this bit right there, this bindIp. +04:02 So the bindIp is set to 127.0.0.1, we should have our firewall turned on anyway, +04:08 we shouldn't be exposing this port, but we're only listening on the local host, +04:13 I think this machine actually right now has two ip adresses, +04:16 one public ipv6 and one net ipv4 ip address, +04:21 but it's not listening on either of them because of this, right. +04:24 So it's super important that this is here, +04:27 otherwise if someone can get to this port on your machine +04:30 and you don't set up authentication, bad things happen. +04:33 All right, so make sure this is here, +04:36 and only change that on your staging servers +04:38 and other things where you control the network, much more carefully. +04:41 Again, we'll talk way more about this in the deployment section +04:44 but for now, this should do for our development environment. +04:47 The other things we'll have to do is we want to set up a RoboMongo, +04:50 we want to set up PyCharm, and we want to make sure +04:54 that we have Python 3 on here, I believe we do, 3.5.3 is more than late enough, +05:00 so we don't need to install any Python, +05:02 but be sure to get RoboMongo and PyCharm, +05:05 whichever version you want set up so that you can follow along. \ No newline at end of file diff --git a/transcripts/ch3-computer-setup/7.txt b/transcripts/ch3-computer-setup/7.txt new file mode 100644 index 0000000..0bcf2e0 --- /dev/null +++ b/transcripts/ch3-computer-setup/7.txt @@ -0,0 +1,23 @@ +00:01 Let's quickly review how we installed MongoDB on Linux. +00:04 So the easiest way to do this is just go mongodb.com +00:07 go to the download button and pick in the little drop down your distribution +00:12 and that way you can copy along. +00:14 Here's what we copied and ran, +00:16 so we visited the Ubuntu setup page at mongodb.com +00:19 that's what I just described, and then we ran add key +00:22 so they said look we're going to trust +00:25 this particular source that we're about to give you, +00:28 and then we said I'd like to go to the repo at mongodb.org +00:32 and set up this, basically pointed at this package store here. +00:38 And then, we're going to update, don't forget this step, +00:42 make sure you update your local listing by pulling from all of the sources, +00:45 and then we're going to apt install mongodb-org, +00:49 and that will take all of the tooling and servers and client stuff; +00:52 if you only want one of them, just the sharding server +00:55 or just the server component, you can absolutely just install that piece, +01:01 we saw like listed on that first page I described +01:04 that there's actually this mongodb-org, +01:07 it's kind of a meta package of four other smaller packages. +01:10 And then at the start we just say service mongod start, +01:14 if you want to change a configuration it's at etc/mongod.conf. +01:19 Change that, just restart the service and it should pick up the changes right away. \ No newline at end of file