Scale By The Bay 2021 : Jennifer Reif, Pouring Coffee into the Matrix - Building Java Apps on Neo4j
thank you very much um so we are going to dive right in i've got a fast-paced bit of content here uh with some live coding coming up so i want to kind of get get through content here so um i'm going to talk about building java applications with neo4j um and my contact information is down there in the the lower left hand corner of the slide i'm almost positive it's on the very last slide as well so don't don't uh you know rush to try to jot that down it'll be available at the end as well and i'm happy to share that uh whoever would like that if you forget later um but please do feel free to if you think of something even past the q a chat here later feel free to contact me um any of those methods there are are good i'm reachable um and so i'm happy to continue the discussion or answer questions at any time all right so just a very brief background on me i'm a developer um and i like i enjoy learning and then i enjoy sharing that learning in a variety of ways whether that's through blog posts or speaking at events like this um and then of course i have several other hobbies as well i love you know cats and coffee and traveling and star wars and all those wonderful sort of things and they basically all boil down to i'm a geek in one form or another and i kind of enjoy that so that's just basically me in a nutshell all right so let's talk about what actually is graph um this is a very formal definition don't get worried if there's a lot of you know kind of confusing language there or you don't quite comprehend all of that that's fine we're going to walk through all of this and hopefully here in just a few minutes you'll have a much easier understanding of this so graph is formally a set of discrete objects each of which has some set of relationships with other objects this was actually it's not new even though graph technologies are relatively new this whole idea of graph is not new it starts all the way back with leonard euler who was a mathematician scientist and and many other things back in the 1700s uh and he wanted to solve the problem of these seven seven bridges of conusburg uh in the city of conesburg prussia there were four land masses connected by seven bridges as you can kind of see in those two left images uh there at the bottom um and he wanted to to uh answer the question can i access all four of these landmasses cross each of the seven bridges and come back to the starting point without duplicating um and so this uh study and research uh became the foundation of what is known as graph theory um so you have uh kind of entities or blocks in this case land masses that are connected via relationships or lines pathways basically so uh graphs then show how data is connected uh for instance we have this this image here we have people entities in blue we have uh kind of technology entities in green and then we have company entities and kind of a reddish orange there um and take for instance one entity there the center node entity is jennifer which yes it's named after me um but we can get a better understanding of jennifer's network and the things that jennifer's world operates within um by looking at the relationships and then the relationships of relationships and so on out so you have several people she's connected to in fact not just connected to but is friends with um she likes a few different technologies she's connected to one company but then we can start see okay if we go out to her friends what companies is she connected to to hops out uh why what might be you know potential um technology she might use in the future or things people might introduce her to et cetera based on the things she's immediately connected to right now so uh just taking some couple of really silly examples and then a couple of business more business related ones um have you ever walked into a room forgotten what you needed left a possession somewhere you can't remember where written a code or requirement forgotten why you even did it um and then taking kind of maybe some more business application uses use cases um not sure where a bug or an issue is coming from so you're trying to track that down you don't know the impacts of a device or system being down you're unsure how your customers relate to your orders or maybe your employees or or who knows what or the products they purchased this is where graph comes in this is where graphs excel it helps you trace steps it can document a pattern or a pathway not just the data at a particular point in time it tells you how and why something might be the way it is or how it's operating not just who what and where um it helps you understand the connections so um what are maybe alternate routes or impacts based on uh certain pathways in my network maybe those pathways aren't available anymore because of x circumstance okay how do i understand better other options that i have this gives you better assumptions and better decision making in businesses because uh obviously if you have something that these patterns maybe you want to explore one pathway or another pathway or like i said something might go down and you need to explore the alternates it gives you better options for better decisions because those paths are already there they're already being mapped out so let's take just a quick example here let's say we have you and this person you might want to become friends with now you can become friends with this person for any sort of motives you might want to sell them something you might want to mimic their success you might want to just be a good friend or or just want to connect with this person because you think you have a lot in common but how would you go about becoming friends with this person well probably without even probably thinking about it you have these things that you like to do or things that you enjoy or topics you know a lot about et cetera so you kind of have this little mini graph around yourself of things that make up things about you and then when you want to get to know this person you somehow interact or talk with this person and you'll find out things that they enjoy and they may not always overlap maybe there are topics you might want to learn more about or maybe they're topics that don't interest you at all and then you'll come across things that you actually have in common so things like music coffee pets and food all of those are things that i really enjoy and hopefully my prospective friend would as well but this gives me then topics and connections and relationships that i can build upon because then okay they like music maybe i might want to give them concert tickets or maybe i want to go to a concert with them because we both enjoy this certain thing maybe i can sell them coffee better because we both understand coffee we both enjoy it i have i have a deeper connection maybe a better relationship with this person because we both like and know a lot about this topic or maybe since we both like food maybe now we can go cook a meal or they can show me how to cook this or i can show them how to cook that or maybe something to go out to dinner and try a new restaurant in the area et cetera so all these relationships anything that you have connections with then gives you better ideas for other things things you can do things you can buy things you can help them with let's take another example so you come home from work at the end of the day you're out on your balcony maybe sipping a nice cup of tea or enjoying a piece of chocolate cake and you're trying to remember where in the world that i set this thing down um maybe i had a you know a bag or something my work bag or or what have you that i sat down and i can't remember where i set it down so what what you typically do is you start tracing back through what you've done okay i went to the closet to put hang this up okay i went over to pet the dog near the kitchen table i came in the door oh i dropped my bags when i came in the door um because i had my hands full or my dog was jumping on me or maybe i had to answer a phone call etc i dropped everything right at the door that's how i know where this was so again tracing that path figuring out maybe what occurred before when or why something might have occurred so let's take something a little bit more serious now those were kind of more fun use cases but in business graphs are being used for things like social networks so understanding how people form relationships or maybe how those relationships break apart and this doesn't have to be necessarily social media it could be a workplace it could be teams it could be lots of different things fraud detections not only understanding users but now also their financial data around them and trying to figure out what are abnormal patterns or unusual things that people are doing with their finances that maybe look suspicious so for instance and in this small graph here you have a few users sharing social security numbers that's obviously a red flag but then you might have some users maybe sharing an address but then passing thousands or millions of dollars back and forth that also looks suspicious so analyzing users their financial habits and then trying to see what are the abnormal cases that you might flag as project potentially fraudulent um this one i like because it's a physical representation of a graph so networking and it operations this is kind of mapping out your devices over maybe a building or a campus or a home or any place so understanding how physical devices are connected with physical wires when something might break or need upgraded the status of everything and maybe kind of an overall view of or map if you will of of your data it devices and then identity and access management i find this topic very confusing anyway i've had several cases uh where it's been like oh i have uh i have this particular role i have access to this resource um but why do i have access to this resource where did this come from and so graph helps you trace back through okay this role was granted by this person this role has direct access to these three things oh but this role also has access to these three things plus this other thing and so graph helps you trace back through that and understand where this stuff is coming from better and then finally where a lot of graph technologies got their start i was in graph based search so how can i provide better search results better recommendations um through things that users are searching okay so let's look at the actual structure of a graph how do you build one um graphs are being known for white sport are known for being whiteboard friendly um so typically at least when i have a new technology problem or a new domain i'll sit down to a whiteboard or piece of paper and draw it out your process flow what data might be coming in et cetera and that's exactly what i've done here and this is a model of java version data so i have java versions i have diffs between the different versions and then i have kind of sub changes within those diffs so things like packages and classes and fields et cetera in the java language we're actually when we write our our java application we're going to focus in on just the java version and the diff on the left hand side of that but this is kind of the full model that i drew out and then you might formulate that into something a little bit nicer a little bit neater go to a modeling tool or some sort of drawing tool um that's what i've done here you can show this to maybe management or your business users or your colleagues etc and then when i actually get ready to store it in the graph this is what it looks like in the graph database so notice then how excessively similar actually exact it looks to the way i drew it on the piece of paper in the first place so this is what's known as being whiteboard friendly your model looks very similar or exactly the same in the database as it actually does in the real world as you would draw it on a whiteboard okay so how what are the pieces of the graph what's kind of the lingo and all that so you have your entities or your nodes in the uh database which if your mathematical background these are known as vertexes for graph theory these are your objects so kind of your nouns person place thing object idea in your domain then you'll have relationships or mathematically edges and these are your connections so these help give context to how your nodes how your entities are related to one another for instance uh we have our two people jennifer and michael um not only am do i know you know jennifer and michael are connected maybe they just met in a coffee shop maybe they shared an uber ride maybe they went to the same conference no we actually are saying they're actually friends here they're connected via a friend relationship so that gives me ideas for maybe um who i want to talk to or connect with or get resume references and so on because i know these two people have this kind of relationship okay so now let's look at how we would develop applications with this um there's a variety of ways to do this i'm just going to show kind of one that i'm i'm most comfortable and familiar with um and feel free to if this is not your preferred method go out and check out plenty of others we're going to use the neo4j java driver which is an official driver and then we're also going to use spring data neo4j which is spring data's integration with neo4j and that uses the java driver at the base so i don't have to specify anything separate i can just use uh spring.neo4j or sdn as we term it and that includes the java driver for me there are plenty of other options like i said we have plenty of official drivers for various languages like python javascript go.net and there are some others out there as well that i haven't listed here but then there's also plenty of production ready and used community built drivers so partners community users customers have built their own drivers for various languages and those are out there and available as well if you're not a spring fan or not familiar with spring there's other java frameworks that we integrate with too so things like quarkus heal it on micronaut again check it all out there's plenty of them so whatever your preference uh feel free to uh to take a peek all right so the uh one of the reasons i'm using spring uh tonight is that spring gives me a lot of stuff out of the box which makes it really easy to demo uh this evening especially in such a short time spring initializer is one which i'll show here in a minute um spring boot does reduce some of the boilerplate for getting an application up and running which is really nice there's annotation based object graph mapping for your plain old java objects so if you're familiar with orm which is object relational mapping which is basically a framework that maps your database to your application objects this is basically the graph version of that and then spring provides query derivation so some kind of built pre-built kind of provided out of the box queries like find all find one find by id that pretty much everyone's implementation is going to be the same for so they give you those right out of the box but then if you want something specific or you want to tweak those you can absolutely write domain specific language or dsl for your custom queries and since we're using neo4j neo4j's query language of choice is cipher so that's what we're going to see and then spring data has a lot of easy ways to connect to neo4j there's bolt http or embedded methods i'm going to use bolt mostly because that's the standard and most performant for today so all right let's jump into our demo so i'm going gonna hop over to where did my window go oh i love when this happens not one moment there we go okay so um pop over to spring initializer here so it's just start that spring.io and it's gonna give me a couple of uh kind of options so i'm gonna choose a maven project for today java is my language of choice at least for now spring boot this is the standard version i'm just going to kind of organize my project structure for my own personal needs so that we're going to call this coffee matrix and we're going to leave it as a jar and java 11 is kind of the standard and then i need to add a couple of dependencies so the web dependency is going to give me rest api capabilities so that i can interact with my data via a rest api and then i need the spring data neo4j dependency as well just going to generate this it'll drop it into let's just drop it into downloads for now good i can unzip that and open that up with my preferred ide intellij switch over to no it's not wanting me to do that okay well oh there it goes okay still loading that's what i was doing okay so as the project loads i do have a neo4j database in the background um for this uh i don't know why intellij has been doing this to me lately um and i'm running neo4j aura which is our cloud neo4j provision database if you have more questions about that feel free to ask me afterward i'm happy to to talk at length but it's a free database anyone can sign up to get a free database you just need your name and your email address um and it's neo4j provisioned um and uh good for like proof of concept projects and uh hobby projects and exploring and all kinds of good stuff so feel free to check that out okay so i have my project structure here um and you can see this is my palm and it gives me you know java 11. i have the 2.5.6 version of spring boot um and then the neo4j dependency and the web dependency which is going to be my rest api and then it includes a test dependency for me already and then the maven plugin for building everything so that's pretty much all it's going to give me now i want to set up some properties for connecting to the database so i'm going to use the uri and i'm just going to set a few variables to fill this in so that i'm not sharing valuable kind of identifying information here for my connection details be a good citizen and and don't share that stuff right um so neo4j username this is just connect to the database so nothing really fancy here um j password neo4j and choose the database and uh neo4j supports multi database um so therefore i need to specify a specific database to go to uh now i can set up these configurations here to put these parameters in so neo4j uri and neo4j username and neo4j password now doing this i'm going to stop my share for just a second don't worry this is purposeful um and i'm going to slide over and just enter my username password details so that i'm not sharing that with all of you uh but give me just one second um but a little bit on a neo4j aura while i'm while i'm kind of waiting here um [Music] free database there are other tiers and again i'm happy to share that so if you need something you know paid tier more support you know things like that we do provide that um but the free tier is open and available and it's super nice for for this kind of stuff so i'm really enjoying it uh quite a bit so let me switch here back to zoom back to share my screen okay and i'll go back to intellij okay so hopefully that worked okay for everybody all right now we're ready to start building uh some applications some classes here so we have our java version um if you remember the data model we just have our versions and our version diffs that we're going to focus on today so i need to tell spring that this is an entity type in my database so i'm going to use the at node annotation i need to tell it i have an id field and then this is a private string java version field and then i actually did an oops when i imported the data into the database i just named my property version in the database so i have to map it and tell it hey look for version when you go to the database side but i'm calling it java version on the application side so that's all i'm doing there and then we'll have a couple of string fields so name code name for a java version um the status of it and then an api spec i believe that's a link for that but we're going to leave it as a string and then i need to pull in my constructor um and do some getters for all of these oh there we go uh now that looks pretty good now let's go over and we will add our repository class so this allows me um to connect to uh database and have some methods uh that connect and let's see we need to extend the neo4j repository and we have an entity type java version and the id type for that entity is a string now that's actually all we need to do because we're going to utilize the out of the box derived methods that spring provides like find one find all et cetera so we actually don't have to specify any methods oops i need to stay there and go over here now i just need to specify our rest api so java version ruler create a controller class this is the rest controller i need to set up a mapping so a basically an endpoint and then i need to inject my uh repository repo and then it wants a constructor for that so let's do that and then i can specify a particular endpoint now to hit and get specific data back so for this instance i'm going to just use an interval of java versions so multiple java versions coming back we're going to call this find all java versions and we're gonna do a return java version repo dot find all now remember i didn't implement the find all method in my repository class spring kind of provides this out of the box because most people's find all methods gonna be pretty much the same right you're gonna look for all the objects um so that actually puts us into a really good spot where we can run this thing and see what we get back assuming everything's going to operate correctly and i've sacrificed enough to the demo gods um but while that's working uh we'll hop over to our browser that's looking like that's going pretty well oh did it not switch on me i didn't i might have to get out of full screen mode here and go back to oh that's lovely when it doesn't work okay one second alrighty so let's open up new localhost for 8080 versions you can see i've tested this recently and when i get here i'll zoom this up so it's a little bit easier to read hopefully that is better but we have our versions we have the status code name the api specs etc you can see all the way up through java version 18 which actually is not currently released yet we just released java 17 um back in oh and i didn't add the dates on that back in september for that there are a couple of date fields that i forgot to add there but okay with that let's go back to intellij then um you might say hey that's great and all but you haven't built any graph stuff you haven't done any relationships or connections you are correct let's fix that problem i don't keep wanting to do the wrong shortcut okay so let's go over and we're going to do our version diff class now so this is another node uh we're going to use an id this is a generated value because i didn't really have a good business key for this so what i've done just not great practice is i've used the uh internal neo4j id for then mode again not great practice but that's what we're working with for tonight so um and then we have uh different vendors that release java versions and then uh our two vendor and two version here so again we have different vendors and different versions of java we're going to diff between them and so we're going to use all these fields our constructor put in our getter all those things okay great now we still haven't done anything super graphy yet so again we're going to fix that because we need to tie these entities together we're going to do that without relationship and we're going to say from newer and then list sorry version diff that's what we need import and then uh because we're we're coming from a java version we're coming from from a newer java version to a diff um so you're going to be looking at older versions right so older version diffs then we can do the same thing going the other way maybe we want to look coming from an older java version what's up what's new in the in the latest stuff so we have a from older relationship out there as well private list version deaf oops not version version deaf and these are newer version diffs okay and then we need a couple of getters for these guys those okay and this puts us in a good spot then we should be able to rerun our app and i'll be pushing time just a little bit because i want to show one more thing in the demo before we pop back over and i give you guys links for everything um okay that's looking good so far and of course that's not going to cooperate with me so we can fix that okay there we go so let's reload our end point here and now we should get the relationships of java versions connected to all the version diffs now in 1.0 the data that i have that i imported doesn't actually tie any version disks to 1.0 not sure why that is it but it's a it's a data problem um so starting at 1.1 then we actually have some values here nothing older of course because they're not comparing it to 1.0 but you have one compared to version five you have one compared to version 12 etc and we could go on down to something newer or we could look at this a little bit better um let's see there's java version 12 so we've got against 10 against seven against two and on down then the newer ones 12 to 16 to 17 and on and on okay so let's jump back here and do one more small thing um we're going to uh maybe that's a lot of fields coming back maybe you just want to skim this down quite a bit so we can actually write a java version projection a projection class or an interface in this case um where we want to say okay we want to do just the string get get java version that and then let's do a list of we're gonna write another projection so this is basically a nested projection um and we're going to call this let's stiff projection get older version diffs and then we need to write that uh projection we're gonna do that here um so basically i'm just projecting creating what kind of like a view um with a small subset um of properties that i wanna get version like that and then i need to actually use this so i'm going to go over to the repo i'm going to write a custom query so again i since this is a projection the spring derived method or finder method isn't going to capture this so then i can go in and write my own and be very specific so i'm using cipher which is a neo4j query language i'm just going to input some stuff here this is basically the same query that i had before um it's just now uh custom i'm i'm right doing the the writing of the query uh what row select and then uh we need enterable java version projection and we're gonna call this find all java versions so that i have a little bit less typing to do here in just a second okay that looks good let's go back to our controller and so i need to change this to the projection projection and then of course it doesn't read this so find all java versions now it should be happy so let's rerun our application and let's see what we get back um all that is going there switch here pull up the right window and reload this so then i should get basically a viewer or a very small subset of entities here which is exactly what i have i have the java version and i have older version diffs again 1.1 isn't compared to anything older and i didn't pull the newer ones because i told it not to in my projection so um we have 1.2 to 1.1 we have 1.3 to 1.1 and 2 etc you can go on down to you know the newer ones uh et cetera there so all that is looking good and that's pulling back exactly what we want so let's wrap up then um and just show a couple of resources so uh this is this all the source code that i wrote and and demoed uh this evening or this afternoon uh wherever you might be in the world um so feel free to check that out i do also have a couple more links within that repository that connect to more full-blown applications um that kind of include more of that data model that i showed on the beginning of the presentation um if you're curious about spring data neo4j the documentation is there not only does springdale neo4j have an imperative support but it also has reactive code support too so if you want to check out a sample application that i wrote um there feel free to look at that third link and then finally if you're curious about the neo4j database as a service or adb that last link as where you want to go for that and again my contact information down there in the lower left i will be in the chat to answer questions thank you guys so much for watching and i hope to see you in just a minute you