Quantum Errors and Benchmarking Quantum Computers
Recording: Quantum Errors and Benchmarking Quantum Computers
um my most recent work i was at google quantum ai working on their hardware team as a research intern um and as as alexis mentioned i'm the president of the stanford quantum computing association and we're kind of a new association about two years old we've grown rather rapidly to about 700 members with half undergraduates and half graduates and some faculty and staff on there as well and our main focus is trying to build a community at stanford of quantum i guess enthusiasts both people who are new like freshman sophomore who don't know anything about quantum maybe nothing about like in quantum physics but then also like a place for graduates to work on projects open source projects um boot camps potentially competitions um and actually create like technical research there and so this year we've expanded a ton with new committees and a new leadership team and we're hoping to have a lot more i guess activities that deal with like companies and corporate activities where instead of having speakers talk in workshops we have like real connections and projects with so google ibm microsoft um very excited to be here and i hope we can contribute to quantum conversations and learn a lot from you guys as well alex i think you're a muted it's early in the morning uh thank you thank you jeremiah so uh yes you know we hope to get more folks from stanford quantum to join us uh you know as the meetings go on and we want to do join events like a panel or something so when you guys are you know welcome to proposed topics so uh at this point uh it will just you know continue the introductions again like we'll play the game of me trying to go in order of names on zoom not repeating uh them more than once uh if i do apologize if i miss somebody i will ask at the end folks to to say a few words so let you know uh save a few words about yourself what you do as usual and what you know what you hope to to learn uh in english conversations i'll just uh start going down the list uh john stanton welcome hi good morning out west to everybody in san francisco so i'm here from canada so i've been on uh quite a few of these uh quantum by the bay sessions so i'm an engineer at ibm in canada so i work with scotiabank which is a large global bank there is a team at scotiabank that do quantum computing they currently do work with xanadu who have a photonic solution and they're working on some additional use cases with um bbva and some other organizations so we're always just interested in meeting other people so very happy to meet jeremiah and the team and we're continuing to try to spread our wings and meet other folks for me uh i've been in ibm a very long time probably longer than most of you have been alive so um i won't say how many years but a long time uh and my degree is in mathematics from 1974 if that gives you an idea so uh background in mathematics and compsci not quantum but very interested in it thank you geronimo welcome again uh nathan shama welcome nathan uh hi everyone yes i'm a colleague of andrea at unitary fund so i will not take too much time and i'm looking forward to meeting you all it looks like a great community thank you nathan and thank you for all your help organizing and you know asking all the right questions to make this happen thanks appreciate it uh looking forward to learn more about unitary funds from andrea and what you guys are doing very interesting group uh okay uh nick brown hey how's it going um i'm i'm from ibm i have a background in experimental quantum computing i'm interested in quantum hardware and uh you know everything else in general thank you nick and nick obviously spoke last time we have uh the recording is up by the way all the recordings are going on functional.tv which is a free youtube channel and the links are posted on the quantum conversation website so if you if you miss something you can always go back and by this time you should have the the video link there uh ali welcome hello good morning uh everyone uh i'm calling from london uh it's uh i think the second section uh maybe the first thing that i've been going on i really appreciate uh the possibility of actually interacting with fellow researchers and practitioners uh around the world um i have a company called quantum contributions and we are building a i can't say much about it but uh i think i can say uh a two words the fact that we are building the photonically a universal company computer got one of the projects that we're involved in and uh with specific uh design um related to some neural networks and uh thank you thank you ali and uh you have an interesting video set up i see you have like a camera with face recognition square on it and histogram well yeah it's so uh you know i spent hundred dollars not to buy a twenty um webcam so uh so that's what i installed it's all like this i got it i gotta tune it up nice nice nice welcome uh amir ibrahimi i'm here i think you're a mute i'm here i was on mute okay sorry about that uh my name is amir i work at unity technologies in our barracuda group it's uh optimized cpu gpu inference engine for unity and uh just bootstrapping on quantum until i can do something useful for the community thanks sounds great welcome here i am here now let's see andrew troytsky uh let me speak from andrei trotsky because he has some bad connection with uh mike's uh and in this video we have from the same group we are from russian quantum center and we are developing the quantum cloud computing uh platform for access remote access to the quantum computer and we will ready to welcome to join uh so thank you very much for invitation uh so let's talk and discuss what we will see in near future of quantum computing sounds great welcome anton and welcome andrew i hope to learn more about your work let's see uh uh debasis banerjee hi alexi hi everyone this is my second quantum conversations by the base session um i'm a particle physicist based in berlin and um i'm interested in trying to use quantum quantum simulation mostly analog but also digital to solve problems in particle physics so in fact one of the things i'm looking forward to in today's one conversation is the use of mytake so together with emily who's also here in the audience today we are um kind of designing quantum circuits and you know we would try to um you know decrease the coherence and and try to solve some problems that's this is these are things we cannot do on classical computers um so yeah we are trying to learn this method and apply so i'm looking forward to to to these discussions thank you great welcome to buses uh ryan larose yeah uh hi everyone i'm also a colleague of andreas at unitary fund so i'm here to support his talk welcome daniel mills hi thanks for the instruction uh yeah so i'm dan i work at cambridge quantum computing so based in cambridge primarily but we have basis in london and elsewhere um i did my phd in quantum computing and undergraduate mathematics before that and i'm here today to talk a bit about uh benchmarking quantum devices and verification and things like that so thanks for having me great uh welcome dania so we you know we have as usual pretty long day so danielle ii uh is the second speaker so you know and basically you know we start right after we wrap up the introductions with the first talk and then you will go second we can do some some uh general discussion in the middle so you know if you're joining us late uh do not you know leave us until you get to hear daniel speak because i think today's program is really really interesting so we're looking forward to this uh sebastian kassinger yes hey alexi um uh i i run the uh partnership programs for ibm quantum including academic partnerships so um uh we've been sponsoring alexis he gets this uh interesting workshop series up and running so we'll see everybody yes thank you sebastian and uh i you know i think it's really uh great how you know we we have uh such a great program today it just shows you know some of this comes from the introductions from partners like ibm some folks just you know step forward like cambridge and and and and offer talks and i you know i really hope we can continue in that sense right you know we have a things like at least if three or four months kind of potential plan but we have room because we have to talk so again if you want to to give a talk at the next event please let me know and uh we definitely will make it happen uh let's see emily hoffman uh yes hi so uh you heard a little bit about me from devastation as well but i'm a physicist uh close to toronto and i work at the perimeter institute for theoretical physics i'm interested in quantum simulation and uh application of yeah simulating strongly correlated systems that are of interest to particle physics and condensed matter great welcome again voiko thomad hi i'm a student in physics and i just joined from a link to from amir nice where are your students where do you study visuals i'm studying romania at ube great great buenos aires welcome [Music] vessel and georgia good morning alex uh good morning everyone my background is uh nuclear and theoretical physics and uh i'm located in california interested in quantum computing working on uh actually addressing uh practical problems uh using quantum computers for example what is efficient vaccine distribution once the vaccines are available and that's one of the things that i'm interested at the moment if anyone knows something about that so please feel free to contact me great great great to have you again i hope you're safe given all the fire situation so uh great great to have everyone yeah uh let's see i uh i think we covered uh everyone but andrea uh and edward is gonna speak now uh the first talk right so is is um is everybody missing at least say a few words about yourself if i miss anybody uh i think we covered everyone all right so let's then uh proceed uh with our first talk so we have andre murray from unitary funds and i'm really er and nathan uh his colleague as well so i'm really you know uh curious about this organization i see you guys do micro grants and there is a whole research agenda so it's really great to have you with us so take it away andre and tell us all about your work and what unitary fund does okay okay hello everyone so basically the best way i guess is to start by sharing my screen because i have a few slides about unitary fun so you can know what we are doing and i will start by sharing the screen okay so actually the presentation that i'm going to give is about a software library which is called mythic but i also have a couple of slides about unitary fund because i think most of you don't know what it is and it's good if i explain what we are doing so unitary fund is a non-profit organization which is legally localized in the us but it's actually completely remote and distributed and indeed i'm working from from italy as i was discussing before and basically the aim of unitary fund is divided in two main blocks the first aim the first task is to run a microgrant program which gives four thousand dollars to open quantum technology projects so if any student any postdoc or any researchers any researcher wants to implement a particular small scale open source project which could be software could be educational and so on can apply for for a grant of four thousand dollars and i invite you if you have some ideas to apply because it's a very interesting project and on the other hand we also have an internal unitary lab in which we do our own original research to help the quantum ecosystem and for example we are building a library which is called mythic and that's what i'm going to present and and we also do like uh standard theoretical research on on quantum software and and quantum information in general so the the unitary fund uh team is with the people that you see here so me and nathan peter ryan sarah that just joined the team and will who is actually the the founder of unitary fund and then we have also an advisory board which is made of 15 experts in quantum systems and quantum software and they are volunteers they they just help us essentially to review all the grant applications that we receive and they are from all the different companies and academic institutions around the world and plus we also have like a community of people working more or less in this field which are mainly based composed of people involved in these grants so we have already 40 grand winners which are connected somehow with the unitary fund and yeah so this is a very short introduction on what unitary farm is and then uh in the rest of this presentation i'm going to focus on what we did in terms of research and also in terms of software in particular i'm going to present this mythic library so this is the the outline of the presentation i'm going to give a simple introduction to the idea of zero noise extrapolation which is an error mitigation method for quantum computers then i'm going to give an overview of the mythic library which is a python library for doing error mitigation and eventually i will try to give some live tutorial on how to use mythic with the jupiter notebook so let me start with zero noise extrapolation so the one of the main problems of current quantum computers is that they are noisy you know this very well and today there is a very simple error correction method which is cross your fingers and hope that your computation will go as you expect and in the very far future we expect that we hope to reach a situation in which we have a fault tolerant quantum computer in which basically the error is goes to zero but you have also a large overhead in terms of qubits and in terms of uh classical uh post processing which means that likely this this is not going to happen in the next 5 or 10 years for sure so we need something in the middle something which can mitigate errors but maybe in a less rigorous and less efficient way but still can give good results without too much overhead so something that can be applied with quantum computers that we have today and there are many methods which are called in general error mitigation methods and one of these methods is called zero noise extrapolation and this is what i'm going to present today but there are also other methods like probabilistic error cancellation randomized compiling dynamical decoupling quantum optical control and at some point we hope to implement also these other methods in our new library that uh that is called mythic so let me start with an introduction on zero noise extrapolation so the best way is to start with a simple plot like this in which we are plotting an arbitrary expectation value in this case is the probability of measuring a qubit in the ground state and as a function of the noise level actually it is as a function of a noise scale factor lambda which means that when lambda is equal to 1 we are evaluating an expectation value at the noise of the hardware and because of this it means that we are not able to measure the expectation value in this region when lambda is less than one but we would like to know what is the ideal expectation value at lambda equal to zero but the only point that we have is this one so the idea of zero noise extrapolation is to instead of trying to reduce the noise you can increase it because that's usually much easier than reducing it and you can have a lot of more points evaluated for the same observable but at different noise levels these are kind of random fluctuations around some kind of mean value and by the way these are real data taken by with an ibm computer and then what you can do is just extrapolate what is the infer what is the inference for the zero noise limit so essentially you do a kind of best fit and you look for the limit of this orange line for lambda equal to zero and this is your guess for the expected ideal zero noise expectation value so as you can expect uh there are two main problems to implement zero noise extrapolation the the first problem is uh noise scaling so how do we scale the noise and the second problem is how do we infer from these blue points what is the zero noise limit we call this the inverse inference step of zero noise extrapolation so i i just anticipate that these two main problems are uh addressed in mythic which is the library that i'm going to present later uh by two uh specific models one is called mythic zero noise extrapolation scaling emitting zero noise extrapolation inference but later i'm going to explain more about this and now i i just want to explain a bit more what we mean by noise scaling so actually the idea was originally proposed in this paper by ibm in which they assume that the system is described is described by a master equation so a time evolution equation for the state of the system of the qubits which is described by a hamiltonian part which is like the ideal dynamics that you expect from your control of the qubits however you also have an interaction with the environment which is going to destroy the quantum evolution a little bit so to disturb it and that's can be described by a noise operator acting on the state and if you are a theorist it's very easy to scale the noise of the system you you just add a constant lambda in front of of the noise operator and and then you can easily evaluate a noiseless evolution by just setting lambda equal to zero you can evaluate the hardware dynamics by setting lambda equal to one and you can easily for example double the noise by setting lambda equal to two and so this is of course very easy but the question is how can you do the same noise scaling if you are an experimentalist so it's not so easy as it seems so let's see what an experimentalist can do so of course it cannot put lambda equal to zero because you don't have a perfect quantum computer so you always have noise at least at a value of lambda equal to one you can evaluate the the standard dynamics standard noise of the system of course you just run the experiment but already if you want to double the noise acting on the system it's not so easy because usually you don't have control over the over the noise that's the main problem and so the proposal which was given in this original paper by the ibm team was to instead of scaling half the noise you can scale down the hamiltonian so if the hamiltonian is k you you evolve the system with a hamiltonian k prime which is one over lambda k in order to recover the same dynamics you also have to scale the the duration the time length of of the pulses and so in practice you just have to stretch all the physical control pulses that you are sending to the qubits in this way you get a dynamics which is lower and so you accumulate a larger amount of noise and what we try to propose with our theory paper that you see here is a digital approach to do some kind of the same kind of noise scaling but not acting at the level of physical control pulses but acting at the level of a gate of a circuit level so what you can actually program with a standard quantum software platform and the basic trick which is behind this method is called unitary folding which basically consists of taking a unitary g and replacing it with g g dagger g to the power of n and since uh g dagger g is equal to the identity for any unitary operation this object is actually equivalent to the input object in a noiseless scenario however it's it contains more gates and so we expect that when you run this object on the right hand side with the real hardware you get more noise and of course you can apply this trick to individual gates and in principle you can also apply the same operation the same unitary folding operation to the entire circuit in any case you increase the length and the depth of the circuit and so you increase the amount of noise and then the main assumption which is behind this approach is that if you if you apply unitary folding the noise is scaled by a factor of lambda equal to 1 plus 2n where n is this exponent here so in practice you hope to scale the noise by all odd integers uh assuming that the noise is more or less proportional to the number of gates that you are applying into the system and the first note is that you can prove that this assumption is exactly correct for the polarizing noise and it is usually a good approximation but of course it is still a matter of research to understand when this assumption holds and when it's violated and so on but it seems to work with real hardware so it's quite promising uh another comment is that it seems that in this way you can only scale the noise by by integer factors but you can also scale by any arbitrary real number larger than one by simply applying unitary folding to our subset of this of the gates into a subset of the circuit so just to clarify a bit the idea of this of this folding trick here i'm going to present an example which is based on a circ circuit which is composed of a quantum register of two qubits and the circuit is made of a hadamard gate applied to the first qubit and a control knot applied to both qubits and if this is your input circuit you can try to apply a global folding for example with a scale factor of three and this is what matic is can do we're using the the scaling module and this is the output as you can see the the depth of the circuit is scaled by a factor of three it has three times more gates and uh ideally the unitary is the same because this is g g dagger g which is equal to the original circuit but if you run this on a real hardware you expect to have three times more noise with respect to the previous one and there is also as i said before more freedom in the way in which you do this folding for example you can fold individual gates instead of folding the full circuit this is an example in which we fold only the first gate and so we get a scale factor of 2 because the depth of this circuit is 4 while the depth of this circuit is 2. and we there is a lot of options to do to apply this trick and all these options are in this module mythic dna scaling however i have to say a few words about the second part so given this noise scale expectation value how can we infer the zero noise limit how do we do this extrapolation and the original proposal which is again in this initial paper by by the ibm research team was to use richardson extrapolation and if you are not familiar with this technique you can explain it because it's quite simple you can you can expect that your expectation value arbitrary expectation value is a function of this lambda which is the noise level of your system and in principle it's you can expand in a tell your series these expectation values in powers of lambda as and assuming that we truncate this series to an order 3 in the noise level and of course we are able to measure this only at values of lambda larger than one but we are interested in e0 so the value the zero noise limit and the way in which we can evaluate this is to take a few measurements at different values of lambda in such a way that we have a kind of a system of linear equations that and we can solve it for e0 which is our zero noise extrapolation and in this case for example you get this result which means that by simply taking a linear combination of your noisy expectation values you can recover the noiseless limit of course up to truncation errors and statistical errors and so on so this is richardson extrapolation and however we tried also to generalize a bit this approach by proposing a more general inference statistical inference approach which is based on a kind of probabilistic model so assume that we have a kind of statistical model for the expectation value as a function of lambda and also as a function of some model parameters that we don't know for example this could be a polynomial an exponential i don't know and then we we take data so we take some blue points of the expectation value for different values of lambda and we fit the model we try to uh deduce the optimal parameters which best describe our measurements and given the best fit model we extrapolate the zero noise limit and of course depending on the model you can get many different extra polyester extrapolation strategies if the model is a polynomial you get polynomial extrapolation you can get an exponential extrapolation you can also recover richardson extrapolation as a particular case of a polynomial extrapolation with the maximum degree and also you have some freedom in in in the way in which you do you collect the data so you you can use a static fit or an adaptive fit in which you choose the noise scale factors uh uh in an adaptive way depending on intermediate uh results so all these methods are contained in the mythic dna inference model and can be easily imported and used with mythic and the reason why we have all so many methods is that it's not clear it is still a matter of research which method is better which matter works which method doesn't work and actually what we observed from our experiments is that it really depends on the problem but even more for the same problem it can depend on the particular hardware for example this is exactly the same experiment run on ibm london of a randomized benchmarking circuit whose expectation values in principle is one and we see that here this violet point so the exponential extrapolation works very well on the other hand with a righty experiment with the righty backhand we see that the red point is better so like the quadratic extrapolation gives better results so i think is this explains why it's probably good to have a library in which you can easily switch between different methods and check which one is working better for for the particular problems that you are trying to address so here are some kind of benchmarking that we try to run i don't go into the details but we try to understand how different scaling methods and different extrapolation methods combine and what's their performance and they take on messages that it really depends they more or less work very well compared to not mitigating at all but which one is better really depends on the particular noise model and problem so i can i can now present by the way maybe that's a good moment to to ask question if you have on this zero noise extrapolation part otherwise i can go on with the rest i have a question yep so earlier you talk about mitigating for various types of gates and for example you have the single cubic gates as well as the two qubit the c naught gates and then when you discuss the lambda value you say that you can have a lambda value of 2 if you just deal with the single cubic gate but then a lambda value of 3 if you include the 2 cubic gate as well but doesn't the two-cubic gate contain a variety of gates um on the for example with the ibm hardware for the super conducting qubits yeah yeah i completely agree so this was just a very simple example which actually is not a very appropriate one from a physical point of view because actually i was scaling the first gate which usually is much less noisy than the 2 qubit case so this just to explain how the technique works but of course if you are interested in scaling for example only the noise acting on a two on a two qubit gate so if you want to take into account how much heroin you have in your gates you can do it by specifying like the the average fidelity of each gate it's an option an additional option that you can give i see now you can also exclude the folding of a single qubit case if you want you can say you can say to mythic to just consider to qubit gates and this is going to to work better maybe anyway i just want to point out that for example let me show this example here in which you have of lambda equal to three and we are folding a control knot three times and also the hadamard three times and so this means that uh independently on how large is is the um is the noise on the of the control knot or of the single qubit gates since we are folding all the gates right um on average so not on average actually we we are in some sense scaling the noise by a factor of three independently on what is the base level of the noise so if if h has a noise of zero this is still okay because adding three h gates is still equivalent to having a zero noise on a single uh h gate yeah i was very happy with this example it was the then looking at the lambda equals two and you're yes yes i agree i agree yes so if there are no other questions i can go on with the second part which is a more high level overview of the software so that let me start with the standard situation that you usually have when you use uh one of the typical quantum software libraries like iskit circon righty uh usually there is a user interface in which you define your quantum program and then you send it to a kind of backend which is on the cloud which can be ibmq righty or it could even be a classical simulator you could even use q-tip for doing low level pulse level simulations and once you get the result you recover the expectation value so this is the typical workflow and the idea of mythic is to sit in the middle between the user and the hardware so it's a kind of intermediate layer which in some sense is very similar to a compiler on the other hand it works in the opposite direction of a compiler because as we have seen before we are trying to given a program to de-optimize it to to increase the number of gates to to have a larger amount of noise and this kind of scaling is performed by this noise scaling module which given an input circuit produces a list of circuits with with more and more gates so there is a kind of noise scaled copy of the same circuit and all these circuits are then executed with a back can which actually has nothing to do with mythic so mythic just uses it as a black box and so this is the reason why it is compatible with with virtually any kind of backend which is able to execute circuits and then mitig collects the results and gives to the user the mitigated expectation value so this is the structure of the package uh there are some tools which are related to conversions for kiskit circuits by quill circuits and then there is this zero noise module which contains the noise scaling and the inference sub packages so we have recently put on the archive a white paper where you can see more details if you are interested the the the library is already at this disposal at the pi pi um repository so you can install it with pip install mythic or if you're interested in the source code this is on github and just a few information about this library it's very recent so we started this year to develop it and it's we are trying to test as much as possible our code and we use continuous integration on different platforms linux mac and windows and all the functions are documented in this documentation link that you see here and the documentation itself is automatically tested so we try to to make everything as stable as possible this is the roadmap for the future now we have only zero noise extrapolation but we hope to implement probabilistic error cancellation very soon and also to support additional backends like q-tip or open pulse and kisket pulse so again if you have other questions on this general overview we can wait a little bit otherwise i will jump to the jupiter notebook i'll ask a question um do you have any criteria for determining which extrapolation method one would would use uh it's a good question so for the moment we don't have a criteria so basically it's an open research question and the good thing is that it's very easy to switch between different extrapolation methods and so if you're if you're solving a problem for which more or less you know what you expect you can check which method is working better otherwise i agree that it's a very hard and interesting problem to to understand a priori which method one should use but we are working also on this on trying to automatically understand what's the optimal way of doing it so i think we can go to the tutorial and by the way if i'm going if i'm running out of time you can stop it as you want i'm not sure how much time still i still have oh you're good you're good we basically allocate about an hour to talk and questions so you know we have yeah yeah i i did almost half an hour right i'm not sure okay let's see let's start with this tutorial maybe we can stop in the middle if we don't need to complete it anyway this is what you need to do if you want to install mythic just run the command pip install mythic and if you want to install one of the software libraries you can also do it and later you can check if everything is working okay by importing mythic and using the mythic about function which tells you all the dependencies of mythic its version and all the dependencies so the only thing that i want to stress is that here you see my quill and kiss kit but it doesn't mean that you need to have installed all this by quill kiskit library so depending on which one you have in your computer mitig will just use what you have already installed it's compatible with with everything but you don't have to install everything and and another important top issue is that circ instead is used for the internal representation of circuits and gates so this is actually a requirement of metic and so what what we need to do in order to implement zero noise extrapolation we need to do three main steps we need to define a circuit of course this is the what you will do anyway with even without zero noise extrapolation then we need to define a function that executes a circuit and gives you an expectation value so this function is called an executor and it's a very important topic for metic it's it's what encapsulates your hardware as a black box in such a way that mythic can call this black box in a hardware independent way and then you can use one of the functions that are in the zero noise extrapolation module to get the zero noise limit so let's try let's try to do it in this case i make a very simple example which is based on kiskit i import keyskit and we can define a register of just two qubits and two classical bits in which we are going to store the measurements of the two qubits and we initialize the circuit and we apply the gates so the circuit just to show a very simple example is extremely simple it's a 40 control knot gates one as a sequence of control not gates and since they are an even number of gates in principle this circuit is equal to the identity and we expect to find the qubits in the zero zero state but of course because of noise they're not going to be always in the zero zero state so this is the circuit very very boring circuit all control knots and two measurements and now we define the executor so as i said before this is a function which takes a quantum circuit and gives you an expectation value and and then the user is completely free to define this function uh in any way in any possible way and in this case i'll just give you an example so if you use a real hardware and you want to use kiskit you can use the standard approach of executing circuits with a key skid so you have to initialize a provider then you get you have to yes initialize also a backend which you can take from the provider in which case we use this back end here and then we are ready to define our executor function which takes a quantum circuit and execute it with a kisk execute method with the with the back end that we just initialized and an important remark is that we set optimization level to zero which means that we try to avoid too much simplifications too much optimization of the circuits because if you use unitary folding and then the circuit is simplified this kind of unitary folding is cancelled so we don't want this and that's we remove this compilation level step and so then we take the measurement counts and we check how many times we get the zero zero result which is the expected ideal result and so we compute essentially an expectation value and just to to monitor the execution we also print the result each time we call this function yes and by the way since now we are not going to use the real hardware i just defined exactly the same executor with a kind of fake hardware which you can import from from kiskit and this is just to simulate a real hardware execution in a classical way and you can we can just try to use the executor so we take the input circuit which is the one made of 40 control notes that we defined before we execute it and we get an expectation value of 0.8 which is not 1 which is the expected result and the reason is that of course we have noise and the circuit is long and so there is some error here and now we are ready to to use mythic so once you have you are in this situation using mythic at the basic level is extremely simple because the only thing that you have to do is to import the zero noise extrapolation module zne and then you can call the function execute with dna which takes as input the original circuit and the executor which is this function which executes circuits both objects here are defined by the user and now mitig is going to call the executor and to scale the circuit and to produce the mitigated expectation value and for example this is the result as you as you can see it's 0.93 of course there are some fluctuations each time you do it but anyway it's much better than 0.84 and so we are using error mitigation in a automatic way so what what is doing mythic behind the scenes so what you don't see here is that matic is internally scaling the input circuit here using unitary folding scale factors one two and three and then e mythic is calling using the executor with this scaled noise scale circuits three times because we scale the noise by a factor of one two and three and each time you call the executor you get an expectation value and in this this is the print message which is generated by the executor because here we have this print expectation value here so what you see here are the individual noisy expectation values as you see they are decreasing but then you can extrapolate with richardson extrapolation from these three numbers to the zero noise limit and you get this so everything is done in an automatic way however if you want you can tune the details uh with with the much more freedom of choosing the scaling method and the inference method so this is what i'm i'm going to show now for example uh what what i said before in in the presentation is that there are different ways of scaling the noise and all these different ways are inside the scaling method for example here i'm in initializing a noise scaling method which is called fault gates at random that takes a subset of a random subset of gates and it applies unitary folding to this subset then we can also choose an inference method for example richardson extrapolation with the scale factors that we just decide here one two three and four for example and then we can pass these two objects the noise scaling method and inference method as options for the execute with dna the functions that we used before and as you can see we get a similar i mean even better okay it was a fluctuation even better result than before and uh and of course here you are free to choose a different noise scaling method for example here you see that you can choose fault global which is the one that i have also shown in the slides and also the inference method can be changed for example we can use i don't know exponential extrapolation and let's see okay the result is not very different in this case but anyway there is a lot of freedom here you can also change the scale factors you can add another one uh and so on okay this is about scaling method and and inference methods and there are even more details but uh i'm not sure if i should go more into the details maybe i can stop here and wait for any questions and in general on this part or on the previous part okay otherwise i can just quickly show that uh the library as i said you is compatible not just with this kit but you can define exactly the same circuit with zirc or with by quill and all these k noise scaling methods that for example fold global scaling method that we have seen in the slides are compatible with any kind of the previous circuits for example if i use circ circuit let me see no yes as you can see you you can just input a circuit of any type of the supported one so kiskeet by quill and and sirk and they work natively with any type and also here this is important for the question that was asked before you can fold by fidelity so you can also input the fidelity of specific gates so for example here i'm using circ to generate a random circuit which is made of four qubits three moments and as you can see there are some single qubit gates and also a control knot and then i swap gate here which is a two qubit gate and when you when we use a unitary folding function in this case fault gates at random you can set the fidelities of all the single qubit gates to one for example which means that mythic is going to just forget about the single qubit gates since they are approximately ideal gates and is going to apply unitary folding only to these two qubit gates so let's see the result and as you can see single qubit gates are not folded while two qubit gates are all folded and in principle you can also input a dictionary with more specific fidelities for all the type of gates but this is a very handy way of just setting all the qubit gates to a fidelity equal to one and yeah i don't want to go more into the details the only thing that i want to say is that the the good thing of using a an open source library is that you can customize it as much as you want for example you are free to define your own custom scaling method if you don't like unitary folding as we develop it you you can define it in a different way the important thing is that you have to use this a little bit we have to follow the structure of our folding functions but otherwise you are completely free to use different methods and also you can define different inference methods so we have an object which is called a factory which is a class that basically deals with the inference of the extrapolation so it collects all the results and and it produces the fit and the zero noise limit and so by simply creating a subclass of our factory you can define your own extrapolation method so with this i think i can go to the conclusion which is i just explained a general overview of zero noise extrapolation and i presented a a high level overview of the mythic library and also i tried to give some examples with the jupiter notebook and thank you very much for for the attention thank you andre for the great overview [Music] we have plenty of time for questions any questions what is the group most excited about for future research if you all are continuing i suppose you would continue on the project yeah so of course we are always interested in improving these zero noise extrapolation methods maybe try to understand exactly in a more adaptive way which method should be used which scale scale factors should be used in a more kind of machine learning approach uh on the other hand we are also interested in more general error mitigation methods of course we want to implement in the software existing ideas also because that's the first thing to do but we are also interested in developing new methods new era mitigation methods and in particular in the probabilistic error cancellation field there is a lot of freedom and a lot of unexplored research directions that we would like to to take in the future i actually have a question andrea so i you know i'm a software engineer by background right and kind of physicist in my previous life so i'm really um happy to see that you use continuous integration and test coverage which is not something i often see in data science folks so i just wonder you know how you know uh you kind of set it up this way do you have kind of strong soft engineering background in your team and how do you kind of reconcile this two kind of driving issues you know there is one research and quickly putting together prototypes and trying things right in like notebooks who tests notebooks nothing runs in production on notebooks right but on the other hand you have this library which is a software engineering project with test coverage you know with uh github actions so i'm very curious how do you guys reconcile this two directional science and kind of production quality yeah so actually i think by training we are all physicists more or less if i'm not wrong but we try to to do all the best practices of open source software development and for continuous integration we use a github actions so it's a kind of built-in integration tools that we have in github and for testing we use pi test which is also very good for this we use code coverage for yes for for checking the coverage uh well we try to use this the most popular tools for for open source software development and and there is also the experience of of of natan and of wheel and of peter the for example natal work work before on kiskit nothing is here by the way and so some of us maybe not me directly so i'm not are more expert on the research part but there are people in the team with a strong experience on open source and code development even software engineering techniques so we try to do our best great great so you know i have another maybe it's a question for nathan then uh yeah because we have you know a pretty big community of open source developers you know in the bay area and i recently see a lot of interest from really traditional software developers like postgres database operators you know people want to get into quantum and so the skills they have are open source development and maintaining kind of bay area internet infrastructure right and so they have spare cycles they want to play off open source is there a pathway for them to play with metis and uh contribute i mean obviously one thing they can do is help with open source if you have any issues which needs helping with right uh debugging kind of you know purely like refactoring things like you know people can do just being software engineers but obviously i think it's hard for them to just plunge in unless they also learn some quantum physics and so i think the result of people who kind of want to do both they want to use their skills to help with open source and during this learn so i just wonder if you guys are set up for this if you have any cycles for mentorship uh can somebody help unitary fund work as a volunteer in order to get familiar with this field if you if there is any kind of community or you have like any any way like i can send these guys who want to learn and who want to kind of contribute um if you guys are set up to to basically accept volunteers and teach them quantum thanks alexi and yeah i mean first thing i will i will paste here this link uh unitary.fun slash mythic and you can sign up there to get updates on uh the software development on the github repository which you can find at this link and googling it um we're setting up milestones where we are signing issues and this is not just for unitary fund [Music] team but anyone can contribute and also we're like [Music] labeling these issues with good first issues and possibly also issues that do not require a quantum background so we're just starting with mythic so we don't have as many bugs but of course like creativity for enhanced features uh can be can be broad so i can bring as an example in q-tip we do have uh which is another project i'm involved with we do have uh and which i also recommend to folks that may be entering into quantum and quantum physics we do have labels that are code and physics of course on github everything is called but the code label is specifically there to to signify that you don't need the physics background to help there because maybe there is as you said some infrastructure back or something they would like to improve in documentation if you're interested please do sign up at this mailing list i mean it's not high frequency at all and because there are also like updates quarterly updates from the other unitary fund projects and we would like to see uh as much as possible uh collaboration also from beyond the core developers also to other projects beyond mythic and with regard to mythic we're also going to set up other um communication channels so that false can quickly ask us uh or the broader community if i have a bag or if something doesn't work with an installation we're still we're thinking about whether going with guitar or slack and again there's like an open issue about that so we have open issues for discussion and anyone is more than welcome to provide their feedback and so this should help with engaging a wider community and by the way i want to say hi to everyone and i didn't see it before but it's great to see how diverse this community is and the different backgrounds and so and i think it was also a great talk by him there yeah absolutely thank you nathan really appreciate it so the link is in the chat everybody can see that uh united dot font slash mytic and we'll share it on the on the website obviously so people can sign up yeah this is great i mean this is exactly uh where i hope to see you know uh our community going because you know there are common issues we can work on and can send folks to that will you know be a good learning opportunity for everyone so thank you nathan uh this is great uh i want to actually uh welcome another a new um member of the community if uh i'm sorry can i ask a quick question yeah yeah um i wanted to ask andrea if it would be possible to share the uh the the python notebook uh because you know like with emily as she said we are actually implemented the quantum circuit and we have results and essentially we want to see how good these how well these error mitigations work and uh and your notebook is is is it has explained things very nicely so i was wondering if we can if it would be possible to share so at the moment is not public but we can we can think about this probably at some point we are going to add notebooks in the library but uh anyway of course you can contact with by mail and i can send you and and the same kind of functions that i used are also in the documentation so i'm not doing very special things in this notebook so great yeah my suggestion is first maybe check in the documentation but if you are really interested in this notebook i can just send you you can write me an email and okay thank you very much thank you the buses so i want to introduce another member who joined us recently uh he's married essencon and he also comes from stanford quantum so he works with jeremiah on driving that organization marriage if you're here maybe you can introduce yourself and say a few words about yourself yeah of course i am a master's student at humphrey um i did my undergrad also at stanford in i was a double major in physics and symbolic systems so symbolic systems is a stanford specific major it's pretty interdisciplinary uh but it's mainly computer science we also take some psychology and philosophy people usually end up focusing on ai or cognitive science and now i'm doing a master's in management science and engineering but what i mostly took was stuff related to quantum computing right like a ml optimization and also i was a ta for the only quantum computing class offered last year um now i'm an intern at qc where so i am working on uh real projects such as optimization using um d-wave annealers uh it's it's a super fun video then i'm going to get great thank you mark welcome and we're looking forward to uh basically continuing partnership with thompson quantum cmo folks both in the audience and also helping with the panels and and events themselves and obviously when the world opens we all want to go to stanford and enjoy kind of physical meetings as as we used to so welcome yeah thanks okay we are trying to uh we're also i i think we talked with you about uh about selexi we're also like opening up this to berkeley berkeley students as well uh and uh you you'll probably be seeing more people from stanford than berkeley uh in the following weeks great yeah we're looking forward to that for sure sorry i have a quick follow-up question for nathan andrea um in particular at the at stanford association we have a new initiative um where we're pairing stanford students um with open source projects and like pretty much like we're trying to like get people both interested who don't do quantum computing but also those who have much experience want to just do something in terms of simulation or error correction um so we're pairing them with like companies or in general like repos um and we usually have like one engineer from like that repo coming helping out once a week do you think motif would uh meditate would be interested did you guys were interested in such a collaboration or [Music] well in general i think i think so but i i give the words to to nothing but i guess this sounds a very interesting opportunity yes so is it like you having every week some uh some new guests so yeah exactly we could reach out to that community yes and again uh amir shared a great link for mentorship which is uh the quantum open source foundation is also helping pair uh folks uh with uh i mean mentors and students or apprentices as you call them and uh yes right right now we're uh also deeply involved with the review and mentorship of the united fund micro grants so personally i would not have time for like continuous mentorship but if it's like you know one off or we can rotate and we can come and say hi to be great oh that's perfect yeah thank you and yes and i'm sure you know also about this but another way to get mentorship on projects that folks care about is joining a google summer of code program which now has a couple of quantum oriented uh projects out there but i know you know about that thank you i need i need to rush thank you it was a great to meet you all thank you alex in sebastian thank you thank you nathan yeah so you know absolutely guys uh we uh we hope to facilitate kind of the follow up from this so maybe we should also set up a slack or something for now i will share the contacts for magic and we can take it from there and thank jeremiah for asking this i think that's exactly what kind of collaboration we want to see our second talk uh is from daniel mills from cambridge quantum let's see daniel yeah so daniel you can present and uh take it away thanks just um grab that presentation okay is that coming through yep all good perfect okay um yeah thanks very much so yeah so i'm coming from london at the moment i work for cambridge quantum computing um so where it starts up based in cambridge and we work on quantum software um but that um so we yeah so one of the main products we produce is a software library called tickets which um facilitates um quantum compilation and circuit design and things like this but yeah so i'm not let's talk about that i'm gonna talk a bit about some work i did a bit before joining cqc actually um and it'll be about benchmarking their term quantum computers yeah so some of the things i'll cover so why should we do benchmarking in the first place what do i mean by benchmarking what are the challenges in benchmarking near-term devices i'll explicitly talk about quantum volume which is a benchmark introduced by ibm which many of you will have heard of discuss some of the advantages disadvantages discuss some approaches to addressing some of the disadvantages of quantum volume and briefly also cover what benchmarking will look like when we have much larger devices so first off um to get to why we want to do benchmarking it's um this leads nicely from asking the question why are we worried about near-term devices so these nist devices everybody will know about them but so nisk is kind of like this era where we have several qubits possibly enough to do things that we can do on classical computers but not sufficiently big enough that we can do many of the really important really popular tasks like factoring factoring um and things like this so this is the kind of regime i'm talking about in this section and we're interested in these nisk devices for two reasons primarily so firstly is that they provide this a proof of principle demonstration that quantum computing is possible this is for example [Music] referred to using terms like quantum computational supremacy which is a demonstration of the implementation of a computation which couldn't be performed on a classical device and also and possibly more importantly for the talk this present talk and these nist devices help us to understand which technologies perform best and things like this and help us move forward into uh the fall tolerant era yeah so quantum computational supremacy will come up a little bit in this talk um but it's just as i said um the implementation of a computation that on quantum computer which could not be performed on the classical one and the second application i mentioned of near-term devices is understanding how we can move from these noisy um intermediate scale devices a few qubits the cubes are kind of noisy the connection is kind of poor so um how should we move from these devices to these highly connected noise-free devices um and building these small devices helps us understand which software is best for example which qubit technology is best which connectivity should we target and things like this so these are the two um applications of near-term devices and this latter one is of interest to us because once we build these near-term devices we can perform benchmarks and understand how they're performing to see which one which design is better and which will enable us to move into this fault tolerant regime so um that's a bit about um why we want to do benchmarking um now i'll just spend a little bit of time thinking about what i'm really get into the details of what i mean by benchmarking so specifically a benchmarking should ask something like is my device on track to be useful so this is this idea that i mentioned about moving from these nintendo devices into folder at once so is my device i'm building on track to be becoming these fault devices so why why might the device not be useful well of course it won't be new to anybody that um quantum current devices are very noisy so when you're performing some perhaps some unitary operation when you might expect it to result in some kind of rotation of the qubit it might result in some other kind of rotation okay so this is just some indication of noise so yeah so this is why a device needs a device might not be useful so the question then is for example is is benchmarking should benchmarking just tell you about the gate fidelity would that be enough well probably that isn't enough because um yeah so so there are there are means to gather this kind of information so like like randomized benchmarking or topology and things um topography for example they you can learn this type of information but is it really enough probably not because what you really want from a quantum device is not just a bunch of um [Music] like a single qubit so a well tuned single qubit might do well at some benchmark which is just equivalent to measuring k fidelity what you really want is your benchmark to include um the number of qubits you have so you want your a a device which performs well at your benchmark to have many qubits um as well as having good game fidelity so is is benchmarking just the number of qubits times the gate fidelity then well also probably not because not only do you want many qubits but you want them to be well connected to each other um so a well-connected architecture avoids you having to move qubits around a great deal and things like this so here's another consideration so is benchmarking then just gate fidelity times the number of qubits times the vertex degree of your architecture well possibly but there are a few things missing from this um so there's this kind of subtlety that actually the depth of the circuit you're implementing depends heavily on the compiler that you're using to map your circuits to the device this in turn will impact on the noise and your device and the noise should certainly impact on the benchmark so is it right that actually your benchmark should include the software that you're using um yes so it probably is actually what you're hoping your benchmark will ask answer is is the system i'm building on track to be useful so the point i'm trying to get across with this little introduction is that when you're building a benchmark it should um [Music] give you an idea of how all of these things are performing collectively so you want to know um you want the benchmark to include the performance of the software to include the number of the qubits the quality of the gates and the connections and the number of connections things like this and these types of benchmarks which include a whole variety of different things often referred to as holistic benchmarks um and i might use this term often in the talk so what are the challenges of building a benchmark um so in fact it's kind of not totally clear if when you have these very large devices that you should be able to really benchmark their performance so um i mean but by the very nature of um quantum supremacy the computation that you should be performing should be impossible to reproduce classically so the very simple type of benchmark where you just perform the computation on a quantum computer and then reproduce the bench reproduce the computation on the classical computer it should be impossible if you believe in if you believe that um quantum supremacy is a real phenomenon so the main the main fear you might have when thinking about benchmarking is that actually the only way to benchmark a quantum computer is to use another quantum computer to check the results um so yeah so that this is this is the fear that you might have when thinking about benchmarking fortunately i can give you a very simple um counter examples this uh factoring for example so um famously quantum computers are able to um or a large fault tolerance quantum computer would be able to factor a large number into its prime factors uh so a benchmark that you might think of doing is to give a quantum computer a large number and ask for its prime factors and if it does this correctly then you can say probably the corner computer is pretty good so okay so we're kind of saved actually it turns out there are some problems where you can perform some kind of benchmarking um just as a classical person you don't need to fill another quantum computer to still take this unfortunately we have this consideration which we introduced earlier that we do we don't have um fault tolerant full fault tolerant quantum computers so in particular we can't do factoring but perhaps there is some other means of cheating the system so let's just try and build such benchmark okay so i'm gonna divide benchmarks into two components so one the first component is the circuits to run on the quantum computer the second is the measure we use to judge if the circuits were run successfully so in fact the factoring example i mentioned falls into this kind of division so the circuits we're running will just be circuits required for factoring and the measure of success we'll use is are the um numbers i'm do do the numbers i'm receiving back multiply together together to give the number which i sent to the quantum computer so it's kind of a binary thing yeah if they do multiply together then that's great um if they don't then that's terrible so so so this problem this example divides into these two divides neatly into this division i'm enforcing okay but we exist in this world where we can't do factoring so let's um think a bit a bit more about this so the circuits i'm going to run instead are um so random circuits and the computation that they perform is random circuit sampling so these problems are very popular at the moment um because it turns out that this process which of random circuit sampling which i'm going to describe is hard to do on a classical computer and it was these types of circuits for example that google ran in a supremacy demonstration so i'm talking now explicitly about quantum volume so i'll talk about the random circuits that are used in that case um so recall quantum volume is a benchmark introduced by ibm and the circuits used there are this form so you have random two cubit rotations which are i'm denoting by these red squares and then you have these these blue squares which denotes permutation layers so inside these blue squares you should imagine many many swap swap gates so the cubes are just being swapped around and then there follows another layer of um two cubic random two qubit unitaries and swaps and um then hey a layer of measurements uh so let's pause for a second and understand why these are good circuits to run so if you were to perform do a good job of running this circuit then it would show for example that's um so these these are these random two-cube unitary gates um being able to implement these shows you have a um the gates you're able to access are quite quite varied into high quality if you do a good job of performing this swap layer then um either you have a very well connected device and you can swap very quickly or your saw gates are very high quality um because everything is kind of random you're sort of benchmarking many elements of the device at once so these kind of circuits are nice because they test several features of the device at once so performing well at implementing these devices would test several important features um yeah so that's why this is a good choice of circuits using the benchmark so random circuit sampling is just the process of implementing this circuit forming measurements and the measurements you get will just be zeros and ones so they'll just be binary strings so with each run of this um the circuit you'll get a new binary string so over time you can um if you conduct the experiment many times you'll eventually imagine just counting the number of times that a particular outcome is seen um so of course over time over many experiments you build up this distribution of um [Music] the outcomes that you've seen um which corresponds to the probability of the outcomes in the um [Music] the i the kind of ideal distribution um yeah so how do we measure the success of an implementation of random circuit sampling so let's think a bit about what would be an example of a bad implementation of random circuit sampling so if you were implementing the circuit and it was really noisy then the tendency would be for all of the outcomes now start um being produced with an equal probability so it's just becoming kind of a random uniform noise um unlike in the ideal case where you have some outcomes which are very likely and some outcomes which are unlikely so there's these two contrasting uh cases the ideal case where you have likely some likely outcomes and the really no very noisy case where you have no particularly likely outcomes and it's just kind of uniform so what more attempts at um deriving a measure of success might be to just perform random circuit sampling perform this kind of bucketing of the outputs to see their likelihood and then checking if the distribution of outcomes that you get is close to you to the uniform one or close to the ideal one so this um has some problems so for example um in order to accurately check which distribution is best matched by the distribution that are produced by the experiments you would have to produce very many you'd have to perform very many experiments so um actually it would be exponentially many because the number of possible outcomes you get would be exponential and to check if the outputs which are unlikely um in the original ideal distribution occur uh you may have to wait a very long time or to see the probability of that happening and it may be an exponential number of experiments that are required to do that so this approach of counting the number of times each individual outputs occurs is not going to work unfortunately but what might work is if you um instead just divide the distribution into those which are very likely and those which are unlikely then if you count the number of times that the i likely just likely outputs occur and the number of times the unlikely outputs occur then it'll give you an idea of the accuracy because in the ideal case you would expect there to be the likely outcomes to occur very often whereas if you have just converged to the uniform noise case then the number of times the likely outcomes occur um is equal to the number of times the unlikely outcomes occur um in particular would be they would occur half the time so to introduce some terminology the outputs which are likely in the original distribution are called heavy outputs um so what we're really checking here is the probability of heavy outputs being generated and this is called the heavy output generation problem or hog which you may also have come across so the check being done is whether or not heavy outputs do actually occur with high probability so this gives us a kind of continuous spectrum in the case where you're producing the noi noise free distribution the heavy outputs occur very often actually because of the nature the form of the distribution in the random circuit sampling case you can calculate the probability you expect to see heavy outputs to be roughly 0.8 in the very noisy case everything is produced with equal probability so the probability that the um outputs which are likely in the ideal distribution are produced is just 0.5 um as with any other division of the distribution so we have this continuous range right in the middle you can imagine some where some examples whether heavy outputs are produced pretty often but not as often as they should be okay so this is the metric we're going to use um to measure success the probability of producing heavy outputs um and this gets around the original problem we had because um [Music] because the there aren't there should be very very small probabilities as um as that was occurring in the when you bucketed according to every output so instead of checking exponentially many buckets you're only checking two now so uh i skimmed it i ignored a couple of problems um which aren't necessarily um conceptually important but i'll mention them so calculating the value of um the kind of halfway point which would be like calculating the median probability of an output um you could imagine would be very challenging actually it can be done the other problem is deciding when you get an output which of the buckets it belongs to which is more challenging it actually requires calculating the probability of that output having occurred in the ideal distribution and checking if it's higher or lower than the median value uh this does take um [Music] exponential time to do but you don't you only have to calculate this value for the number of for each experiment which could be a polynomial number of experiments so rather than having to calculate um probabilities an expansion of probability is an exponential number of times you now only need to calculate um you only need to calculate a polynomial number of probabilities each station exponential time so it becomes a bit easier so yeah this um these are the prerequisites required to understand the measure of quantum volume i'll just pause in case there are any questions um i don't know if i can see the chat yeah feel free guys to ask questions in the chat and speakers can pick them during this talk when it's convenient or just ask a question when is a good time okay can i can i ask a question on this method so to me it looks very very similar to the google quantum supremacy experiment right this kind of checking the results of a random and in that case if i'm not wrong they use kind of uh linearized entropy or a kind of figure of merit to check if the result is what you expect and how different is is this method to to your methods um yeah so so the method i described here i i have one claim is this so this is just quantum volume that's the introduced by again so this is not introduced by me but um you're writing the google case they use was called course entry benchmarking which is very similar it relies on checking how um the pro the how often the likely outputs occur um yeah it's it's it's kind of subtly different i just introduced quantum volume because it's a bit conceptually easier to understand but it is in that case they explicitly calculate the probability of um the average probability of the outcomes which occur um [Music] rather than calculating the probability of the heavy outputs occurring so it is very is very similar um but i just described quantum volume because it's conceptually very easier yeah okay okay thanks okay so so yeah so these are the prerequisites on soundcloud volume so quantum volume um what is that exactly so you can imagine um if i built random circuits in the following way um so if i had two qubits then i would have um i could have this small gadget which is just the rotations and the swaps twice and then three qubits i would have the rotation swap layers three times and then in general i would have n layers of these rotation swap pairs and you can imagine that it would get harder and harder so as i as i as n grows the noise would increase because i have um deeper and deeper circuits um and as the noise increases the probability of producing heavy outputs goes down so i would um [Music] i so the definition of quantum quantum volume is basically the biggest largest n for which you can you're still capable of producing heavy outputs with a relatively high probability um yeah so so as i described as the number of qubits grows the depth of the circus i'm choosing to implement grows so the probability of producing the heavy outputs should fall because it gets noisier and noise more noisier and then when it cross when the probability of producing heavy outputs crosses some value which is two-thirds we call that number of qubits the quantum volume um actually you'll often hear quantum volume quoted as two to the power of that number of cupids so for example there was recently this demonstration of corn volume 64 by ibm and honeywell um which is actually means um like a implemented six-cuber random sentence so yeah so i mean just to reflect so did it do what we wanted it to do well indeed you you needed to have um the sword player and still ensured you had good connectivity um in general because the circuits are quite getting quite deep um you required you were required to have good fidelity uh as obviously as n grows you need more qubits so it checks this it doesn't really explicitly acknowledge the software you're using the um but but yeah it does a good job it's a nice nice benchmark to choose to use so this is um some examples from four ibm devices so indeed you see on the x-axis here the number of qubits and then you have the heavy output probability on the side here on the y-axis so you can see as the number of cubits increases you do have this fall-off and um so the quantum volume nicely allows you to compare across these devices um so yeah just uh some quite obvious conclusions like so higher up is better so higher up is the probability increasing heavy upwards higher up is good so you have orange the orange device in ibm is doing doing pretty well so let's say our instance may be the best device as measured by this metric so in this case you're only seeing heavy out the generation probabilities above two-thirds for uh three qubits so that's like a quantum volume of eight being demonstrated in this case um yeah the column volume 64. experiment was done on a different device um yeah so some pros and cons so um it kind of does what we wanted it gives us an idea you can compare devices and start thinking about which devices perform best um it's um relate so it's related to quantum supremacy i haven't really discussed that but it you can draw some confusions about that if you perform well um but it's missing a few things we might like so it doesn't necessarily give a great indication of how to improve the device quality um it doesn't necessarily give an indication of which applications the device might be better at implementing and it's it's kind of limited to only a few cubits so um as we've seen from this like the best we've seen is one volume 64 so we're just tackling six qubits um [Music] even though we have devices publicly available which are you know on the order of 20 key bits so it has it has these drawbacks so yeah we tried to extend it a little bit by introducing these application motivated full stack benchmarks um which aims to explicitly include the compiler and facilitated the inclusion of what qubits and threats we tried to give more of an idea of which applications perform best on which devices um so including the compilers [Music] can be done quite easily so we just um compile the circuits um using uh five different compilers to check their performance um so yeah so we just have we have these noise aware compilers which are which can perform compilation based on the noise models provided by the device noise unaware compilers and only routing which is kind of compiling just just placement onto the architecture so that it works without any further optimization so for example when you optimize a circuit you can compress things in this case we in the only the only routing case we didn't do any of that just as a check um and you can you can draw some comparisons about which software gives you the best quantum volume um yeah so here here we see bicycle performing quite well um and um the yeah the noise aware by ticket is coming through so you can say that using noise aware placement um noise including noise awareness in your compiler is very beneficial and actually in the recent work from ibm on um volume 64 experiments um they made significant improvements in their compiler and that's what actually enabled them to improve the quantum volume of their device even without uh performing a lot of improvements of the hardware um so this kind of improv when identifying the kind of improvements you can make by using software um can be very powerful um so to increase the number of qubits we can include um you can we chose to reduce the depth of the circuits and you can do that in a way that isn't totally fatal by using what are called iqp circuits so without without going through much detail detail basically they're quite quite simple circuits they just have um hadamards and commuting gates and hadamards and measurements so they can be very shallow and they can be optimized quite a lot because the gates commute with one another and they also have the same combinational supremacy guarantees or similar guarantees to random circuit sampling or at least very powerful guarantees um which is which is nice when thinking about quantum supremacy demonstrations and we also chose to um whereas the random circuits had this permutation layer which kind of implicitly assumed all to all connectivity we chose to reduce the connectivity we assume to at most um [Music] uh three three connections so so there's a mistake in my drawing here that shouldn't be for um [Music] and this makes it a bit easier to compile because you don't necessarily have to swap all the qubits around a great deal which makes the shallow which once again makes the circuits shut up so um yeah we can identify the problem i'm talking about by looking back at the plot i showed earlier so for example by the time we reach so we have um two cubits we can use four key that's five cubits so once the time by time we reach five qubits we've essentially hit the uniform uniformly random distribution so we have this 0.5 value which indicated uniform randomness which means even though singapore for example has something like over 20 qubits we can't really can't explore them because if we just keep adding more and more layers it's just gonna stick to this 0.5 value um so we're kind of stuck a bit but if we use these iqp circuits then even uh for larger circuits so six and seven we can still learn a bit about singapore um so for example if we think about so here we have um singapore which by the time and the iqp circuits by the time you reach um five qubits you're still above 0.5 so you can keep learning yorktown only has five qubits melbourne has flattened out already so we can stop thinking about that um orange only has five cubits as well sadly so but singapore we can keep learning about um which is great um so what about after supremacy um because we've we've said here that um we've used very random looking circuits um which isn't ideal because the circuits we really want to implement are probably very highly structured um so for example this this circuit here is kind of um exemplifying uh what's called the uccsd anzacs in variational quantum icon solvers so this is the kind of thing that you might want to implement to do things like finding the ground states of things like that um and you can see that it's very it's not random it's actually very highly structured so you have layers of um single qubit alligators and then there's these ladders and things like this so the actual circuits we really want to implement for application purposes like in this case would be quantum chemistry they're highly structured um so it might be that quantum volume is not ideal not the ideal metric in this case so actually we uh discovered that if you apply enough of these powly circuits the what are called what we call what we'll call powly gadgets back to back um then you can recreate the kind of um the distribution of outputs that we had before this dropping off distribution where you have some outcomes that are highly likely and some which are unlikely you can so you can recreate that distribution using these power gadgets which is enough to do something very similar to quantum volume and indeed if you implement um these circuits unfortunately they become they have to be very deep to be to reproduce this curve um so we can only explore up to four qubits here but you can start to learn things about these devices and how they would perform when implementing particular applications so you can see for example that maybe you could make claims like orange and singapore do would do pretty well if you wanted to implement some kind of quantum chemistry application so these kind of extensions allow you to make these kind of claims which using quantum volume might not allow you to do um so that kind of solves a couple of our problems so now we can start talking about uh the best software to use we can talk about the best applications and we can include a few more qubits unfortunately um we have a scalar same scalability problem that quantum volume has namely that you have to calculate the probability of the outcomes you see um and that remains a problem um so performing with these benchmark a benchmarking of near 10 devices very efficiently is still an open problem um so yeah so basically we have some possible extensions that might be good to consider so um extending to the beyond the supremacy regime um you might be able to use these kind of benchmarks to measure the quality of noise models um so for example if you're um if you if you could simulate the expected benchmarks and the benchmarks you're getting you might be able to learn something about the accuracy of your noise models uh if we could use these these benchmarks to perform compilation in in place of the noise aware compilers that we used already this could be beneficial um and these benchmarks um allow us to advise device designers and users of the best devices to explore so that would be great to do this one um so i have said most of what i want to say i i will linger very briefly on the prospects for future benchmarking schemes so we've throughout this talk i've lived in this um dynamic so i have a classical person who wants to benchmark perform a benchmarking process with a very noisy quantum small small noisy quantum device so do i gain anything for example if i was able to use a small noisy quantum device to benchmark another small rising common device um so actually you can gain a bit so the downside is you have to have a small support device but you are able to perform a benchmarking procedure efficiently so rather than before we had to calculate the probability of um uh some of the outcomes occurring now you don't need to do that anymore um yeah and the downside is you now need to perform um some qubit communication but but there are schemes which allow you to do this efficiently what about using a small noisy chrome device to benchmark a full fault or on corn device um yep there are schemes for doing that the small noisy quantum device here now needs to be a bit bigger than the one used to benchmark the other smaller equivalent device but there are schemes to do this but the kind of um [Music] what we really really want is to be able to for a classical person to perform a benchmarking of a full fault or a quantum device and what i want is something more than just the factoring example i introduced earlier i want to be able to check the quality of any computation and not just factoring [Music] actually this is a hard problem i was open for a long time but this too has been solved you can check the quality of any computation uh if you're willing to make some uh computational complexity assumptions so if you assume the hardness of some particular computations then you can perform this process as well uh so i end there um it just leaves me to say thanks for everybody's attention and to refer you to um this paper which constitutes a lot of the work discussed and also a lot of the work um yeah so if anybody wants to also want references to any of the other topics i covered in the talk i would be happy to provide so yeah thanks for listening thank you daniel for the great talk any questions [Music] i guess uh it's been a long meeting so if you guys come up with questions later you can follow up with danielle separately i personally really liked it uh so we also had some other topics right we we had i think a couple lightning talks proposed i'm not sure the uh authors are here and generally you know we have a few folks left uh if you you guys want to have a general discussion uh you're very welcome to to raise any topics let's see uh i actually wanted to ask you guys uh about the the format right so we have a settled traditional having two talks uh which basically makes it half a day right uh do you think that uh we should be doing just one talk uh possibly with more general discussion or do you do talks every month is great any any opinion on the cadence from anyone um i i mean i guess since the frequency of these meetings aren't too often maybe that's the reason for the two talks um i find that i would be happy with even one talk for and then maybe more discussion or q a it's just a it's a longer period of time uh compared to other uh meetings i agree agreed that's some of the feedback we've heard and in terms of kind of mixing talks and tutorials i think we we had pretty special interesting tutorials and i guess with a lot of kind of newcomers joining um from different communities uh it would probably make sense for us to to do tutorials so if you guys want to do a tutorial right please let me know so we can obviously do the kind of uh current research reports but we also are very interested in tutorials so andrea like i've seen you know uh you guys have a lot of really good materials and i think ibm has uh learning resources and some of the folks like like nick brown is in the uh kisky textbook right so so if if you guys want to to do tutorial i'd really appreciate it uh and we can share it broadly and i guess uh invite my folks right so let me know if you want to do a tutorial on uh some of the topics you cover right leading to your work danielle you're very welcome to kind of do a one-on-one on in a kind of slower pace i think some of this basically and same same with mitiga i think a lot of this can be unpacked and you know made into a tutorial which can be maybe run separately and maybe with a hands-on fashion with a jupiter notebook experimentation over many couple hours right which we can do differently so um all right well so uh uh unless you have any other questions or comments uh thank you very much guys that was a great great meeting today and really appreciate everybody's participation and questions uh let me know uh if you want to do a talk uh next month and i will uh invite the the list to propose talks we have i think one talk coming from uc davis already so maybe if we switch to uh one talk cadence that will be it but then we'll have more time for lightning talks thank you very much see you next time great thank you all thank you everyone bye bye take great day