SBTB 2023: Salar Rahmanian, Streamline Your Development Workflow

SBTB 2023: Salar Rahmanian, Streamline Your Development Workflow

Recording: SBTB 2023: Salar Rahmanian, Streamline Your Development Workflow

hi everyone thank you for coming to my talk I'm salonan also known as sofo online um you can uh find out all about me and find various means of contacting me uh by visiting my blog um I have been a NYX user and contributor since 2018 today I'm going to tell you all about NYX and how it can help with all of your projects what I want to do is to start with a recap of a typical development life cycle that we all follow to get our ideas from development to production it all starts by you working on your project coding on your laptop I use a Macbook but equally could be something else like Linux once your code is ready for the next stage you push it to GitHub this starts uh the run run of your cicd pipeline job which will carry out a number of tasks for you which may include your tests ultimately after um after a review and release process that you follow your code will end up on a server somewhere for each of these environments you typically have to install and set up a few things let's let's go through them you install an OS which is most likely different between your laptop cicd and production on your laptop you may need some extra tools like G you install your OS and program language packages using multiple package managers typically and lastly you will need to set up some services that your application will need that could be as simple as a configuration to an external service or you may have to run one or more services locally local to your application like postgress or Kafka variations between our various environments and tools have consequences it leads to our work to be non-deterministic and non-reproducible at times just because it builds on your laptop doesn't mean it will elsewhere ultimately it means uh our work is not reliable now let's talk about lost developer time there is a cost money and reduced velocity to deliver your work we spend our time fixing issues due to the non-determinism we discussed we have to spend time maintaining the different environments creating different configurations possibly using different tools to do it then we need to set up the different services our project needs like postgress Kafka in all the different environments and stages of our workflow and finally all of the all of these environments do need to be tested now wouldn't it be nice if we didn't have to do all that and we just had one tool to use for all the environments and all parts of our workflow NYX is the answer and it does solve that problem there are three parts to Nyx NYX is composed of Nyx package manager NYX expression language and Nyx OS let's take a quick look at what they are so NYX is ultimately a package manager it's a programming language agnostic it's one tool one way of building and testing everywhere but many programming languages secondly we have NYX expression language it's a functional program programming language that promotes immutability it's used to add packages to Nyx package manager every programming language has its own way of building a package like how python packages are built is very different when compared to let's say JavaScript or C++ you use NYX expression language to specify how to build it as a user of Nyx you will always build the same way with the same tools and commands NYX will know how to build it accommod accommodating the differences in how to build you use you you use it to tell NYX basically how to build test and deploy your work of course you can use NYX expression language also to configure your services that you want to run for example if you need postest running with some kind of default schema last but not least there's NYX OS NYX OS is is its own Linux OS distribution unlike other Linux os's it uses NYX package manager as it package manager and then of course it uses NYX expression language to configure and set up your OS um SOS you don't have to use nyos as your OS to use NYX though but if you do you will really like it as an example on my laptop I use Mac OS with NYX package manager and use NYX expression language when I need to now without further delay in instead of spending more time just talking about NYX let me show you NYX and how you can use it for your projects it is just a glimpse into the world of Nyx that I'm hoping will get you excited to start using NYX and making it part of your workflow so when I turn my MacBook on I hardly have anything installed let's take a look to see if I've got Python and WG installed and as you saw from from that little clip I saved from my MacBook uh neither of them are installed I need them what do I do let's start with creating a NYX shell I tried my best as you can see to avoid live demos right I recorded the bits I would do live coding um yeah so as as you can see I we we just do Nick shell minus P WG that creates a nck shell for you that now has got WG and conversely when you leave your shell it destroys everything and it's does it in a quite a reasonably fast speed considering I've still got an Intel Mac in for the benefit of uh completion let's just watch it watch watch it do python as well um python as well even though um I think you see you see my demo and there you go the the latest python version is installed and once again when you leave the shell it's it's it's it's it's all gone so wouldn't it be nice if once you are in your project folder you have all of your OS and programming language like python dependen is specific to your project installed and all the services you need configured and set up for you it's kind of like Python's virtual en but with your non-python dependencies installed and services set up for you also in your project you can create a shell. n file that defines how your shell is set up when entering a shell you can enter the name of the shell file to use this way you can have multiple Shale files but if you don't specify it it will purpose NYX will purposefully look for a shell. n file in your current directory and if if it can't find that then it will look for a different called. n file before it gives up now let's take a look on how we Define our shell taking a look at a a typical uh shell. n file shell. n file first we import the Nyx package repository we want to use as you can see it's a tarbo on GitHub with a unique hash for a specific release of Nyx the cool thing is you can create and use your own repository if you want and import it in the same way then in the build inputs array you list the packages you want to install in the Shell hook section you insert any bash commands or scripts you want that is run when your shell first starts here is another example that the we've added a postgress as a dependency to build inputs and we've configured it so that when you enter your shell it starts off uh post for you so now let's see this uh let's enter our shell with this configuration and see it work so first I'm checking to see if postgres is installed on my MacBook it's not now I now I enter my shell uh the file is called shell 2.n so I explicitly tell her to use shell 2.n and off it goes and then we wait a few minutes and then now uh if we check we were able to see that postgis is running now and also the dependencies that we listed in the build inputs part is are there there you go there postest running that shows the background process of postgress rough is installed and WG that I keep using as an example here is is installed and then once again you exit and it's all gone no more postgress and no more uh dependencies there we go there we go no more post great moving on nck shell that I just showed you is great but it's a little bit verbose to set up and can get complex complex for common tasks there is a third party open open source tool created uh by a NYX developer in Europe called de that makes it easier for you to Define your local Dev environment um what I want to do is now take a look at that uh to see how it compares to Nix shell so let's start by creating a new project so we go off and um create a new empty folder we go in our empty folder and we do Dev end in it to initialize the project and there I'm just showing the fact that the folder was empty when I before we did De in it and then off that goes and does a lot of magical Nixie stuff great it's done what you'll see is that it's generated a bunch of files in your project I'm not going to go through every single file uh but let's take a look at the two most important files that it generates for you the first important file that it generates for you is a dev. yaml in this yamile you literally list all the all the Nyx reposit iies that you want imported into a project where you want to install packages from it can be more than one by default it it sets it up for the default NY OS uh NY OS unstable um unst unstable repository the second file that it creates is a dev. n this is kind of like Dev dev's equivalent of the shell

N file we talked about and it is in here that you you define how you want your shell to be configured once you enter your shell so let's go through this uh file that it creates and kind of kind of like uh set it up for a python project as an example so this is the we we we already are importing NYX package manager in the yaml file that it came with what we want to add to here is that Dead's got a separate uh repository that you need to import by programming language so because we're interested in Python here we now add python dev's a python repository as a as a repository recir to the yaml file the next thing we do is that this is the default uh dev. Nile that it generated for you which is basically a skeleton for you to configure it for what you want to do let's alter it and go through it uh how you would use this for a typical python project so you can see on the right hand side um you use EnV to add any environment variables that you want available in your shell when you enter your shell here in this example I've added an environment variable called greet which is simply has a string scale by the bay in the string you add your dependencies that you want installed in your shell to the packages array to the packages array I don't know about you guys but all my projects have scripts in them and lots of bash scripts to automate various things so with dead you can have all your bash scripts inside your dead. file uh you can see where it says scripts. hello. exac hello will be the name of the script file literally is Hello I would write run at the terminal and it will behave just like as if I've run a shell script or a binary and it will run run it and single quotes or multi-line string in front of it you just insert your your bash script basically basically it's as easy as that enter shell or bash terminal commands that run when your shell starts which is great for running any initialization tasks but the exciting part is when it comes to Common programming languages and services you no longer have to add them as dependences and do a lot of verbose configuration like like I demonstrated with Nick shell where you had to configure postgress with all those multiple lines of uh bash to bring it up and everything what you do with dead V is that you literally got a menu of all the Poss all the common programming languages and services that you're likely to use and you literally just enable what you need and override any of the defaults it has in my example here I'm enabling Python and I'm choosing what version of python I want we didn't have to specify the version of python and if we didn't NYX would have chosen one for us typically the latest one that's available in the repository that you imported similarly when enabling postgress we didn't have to list the initial databases it would have defaulted to the postgress database as the only database if we hadn't done that looking at the devm documentation which I'll give you a re Source later you will uh there are many programm services and tools that you can just enable this easily for developer happiness so now let's see the this new new shell that we created with Dev in action and if you recall I I created an environment variable called greet which says scale by the bay and then in my enter shell I print it out and also I print out in my enter shell as soon as we enter sh the version of git so as you can see that now that we've entered the shell it does say hello from scale by the bay it does print the version of git and it does have the the correct version of python installed uh that I specified uh and but when you look even though postgress is installed it's not running straight away what devm expects you to do is to do devm up as a command to bring your services up so now I'm doing Dev up it creates a little kind of like a subshell within that shell and it it will bring up postgress for me Happy Days very minimal configuration so far I have shown you two ways to manage your local Dev environment but I did promise to show you how to solve it everywhere giving you reproducible and deterministic environments everywhere not just on your laptop so let let me show you NYX flakes now what is NYX flakes it's a package manager for your project think of cargo what cargo is for rust NYX Flex is for your P for your for your for your project whatever language it is it has conventions on how to build deploy and run your project it's CL platform and it has a distinct schema for you to follow let's take a look at what a flake looks like sorry if the text is very small uh but I was genely trying to fit the whole thing on one slide um um looking at this NYX flake you will see that you have inputs these are these are the list of package repositories that you want to import your packages from it can be more than one similar to that yaml file that de had you just literally list all your inputs here here you have outputs these are a list of systems you want to be able to build deploy and run your application on looking at the all systems array we are listing all the systems we want to build deploy and run run on which does include both the arm and Intel variations for Linux and Mac OS looking at the packages part you define how your project is built um we choose the version of python we want here I've chosen version 3.9 deliberately so that it's different to the my earlier examples the build input list again includes the list of the dependencies you need and the SRC uh um shows where the root of the your python application is root of your py python application is going further down looking at the dev shells part of this flake this is where you define your local development environment this is same as NYX shell that I showed you earlier to to be consistent with my earlier examples once you enter enter the dev environment using this flake you'll have rough and W get as as your dependencies as specified in the build inputs array and the echo statement that that I've put in there will be printed on the screen once we enter this screen as I've added it to the Shell hook so as you can see within this one s simple flake one simple configuration you can you use which I use n expression language to create I have a single tool and a single configuration that builds deploys and runs on all the systems I listed so everywhere it's everywhere so now let's take let's see how do we use this flake and let's see it in action uh I created a hello world python application for this purpose it's literally one that when you type in a command at the terminal it prints a hello world message for you um I'm not going to go through the code for that as I want to focus on NY uh so let's take a look so you can see there's no python installed and none of his dependences are installed this time we Type n develop when you type NYX develop it will go into the shell that you specified in your flake as your Dev shell an analogous to to the Nyx shell that I showed you earlier you can see that it's got the correct version of Python and the dependenc is Ruff and WG are now available to you next we do NYX run which will run uh the application and print that hello world onto the screen hello scale by the B 2023 and then the last command that I just want to show is the fact is one that's called nyx's flake show which will show you all your inputs and outputs and it will as you can see from this you can see that because my laptop is an Intel MacBook that the only output it's it's attempting to build is the is the Mac Intel one all the other the systems it's ignoring it because that's not the current system it's running on you may ask at this point how easy is it to create a flake for your project well if you NYX provides uh a tool which lets you create your initial flake file um have somewhere to start from uh so basically if you create your new project you go to the root of your project and then you do NYX flake in it NYX will create create the scaffold for your the flake. N file that I had created and then let's take a look at what the contents looks like you can see it's just like a bare skeleton starting point for you to add it add the three different parts that I that that I mentioned to it to the way you want but one step better One Step better is that uh NYX if you go to their GitHub organization they have a repository called templates where they've actually created some default templates for all the different programming languages that you're likely to use uh they're not like anything sophisticated but like you can see that they've got one for Python and um like they got Python and all the all the popular popular languages um the cool thing about NYX templates is that you can create your own GitHub repository of templates that you can use and uh if you search GitHub you'll see there's there's a lot of developers who created their own uh GitHub templates repost that you can use to get started with your project by using one of their by using one of their templates and just to have a quick look at um H how this would work for um um for for for if you if you want to install a template from nyx's own um from nyx's own templates nyx's or the G GitHub or's own template you literally pass the D- template you tell it templates hash python for the for the python example and the reason you do templates hash templates is the name of the repository in NYX organization and python the convention is and python is the name of the sub directory where the template resides and if if if we have a quick look at I've done a screenshot on my next slide I'll just wait for you to see the flake it C with an in with this if you look at the next you you can see that the reort it's NX OSS templates repository and it's the python folder that it goes to fetch the templates from from when you want to do the same thing with someone else's GitHub repo um the steps to use that template is the same but NYX needs to know which GitHub repo it needs to go to and which sub directory it needs to get so the convention it's called at the command is that you specify GitHub which could be gitlab conversely as well then the name of the GitHub org followed by the name of the repo and then hash the subdirectory that you want to to use as your as as as your temp as your template as your template so so far you got to see just a small glimpse of what NYX and Nyx Flex can do to provide you with a single to tool to manage your whole life cycle of delivering your code from your laptop to production the last thing I wanted to share share before finishing my talk was just some resources which I think will be helpful for you to get started on NYX if you haven't used NYX before um if you haven't used NYX before zero to Nyx which is the second URL is actually in my in my opinion the best way to start because it gets you started with your laptop set up with using flake and it gives you enough information to get started without blinding you with science and then of course NYX org.org is nyx's main website where you got links to all their documentation and everything de. sh is where de's documentation resides the Believe It or Not NYX has been around for about 20 years so you'll find quite a lot of content on their nixcon conference uh YouTube channel uh YouTube channel uh I'm not going to read you all the all the links on here uh but Shameless plug the last two links are to my blog and my YouTube clone that's that runs on peer tube uh where there are some content on NY there that you can look at as well and hopefully in future there will be more uh I used to run some meetups in the Bay Area so there's I've had a few speakers who have come come to my meetups talked about Nix and I've uploaded it there for everyone to look for better or worse most of the content is all about G and haskal though but a little bit of Knicks hopefully more Nicks later um more more Nicks later that's all I have thank you all for coming to my [Applause] talk we we've got a couple minutes for questions you all using Nicks now out of Interest Who uses Nicks today fantastic we only have half the other half of the room to convince other half of the room come and find me I'll do whatever I can to brainwash you great so table table one upstairs thank you very much thank you