A Glimpse into our Group Coaching Calls 1

[00:00 - 00:05] - Hello everyone. - Yesterday, I requested access, but I don't know if somebody's able to grant it. - For the Google column, right?

[00:06 - 00:12] - Yeah, for exercise four. - Yes, I saw your push. I texted Mary and she will, she will do the access.

[00:13 - 00:22] - Strange news, I was just gonna say, did you understand my note about the other notebook that had the images in it from Wikipedia? - No, I still, so I went, I went through your push.

[00:23 - 00:26] I did not. So this is, that's the purpose of today's call.

[00:27 - 00:38] Basically trying to resolve or solve like any errors or any trouble that you are getting. It can be your project, it can be mini project, can be like exercise.

[00:39 - 00:42] - Okay. - Yeah. - So maybe what we can do over here is...

[00:43 - 00:46] - You want me to open up that notebook and try it? - Yes. Yes.

[00:47 - 00:52] - Give me one sec. - I think that will give it to you. - Is it tokens and embedding?

[00:53 - 01:00] - Yeah, it was tokens and embeddings. You know, let's start, I'm not looking at it. You find it, dude.

[01:01 - 01:10] - Okay, I'll kind of open where I was running into the issues while long when I was told to use an image. So let me find the embedding.

[01:11 - 01:15] - All right, I'll share my screen. - Yes, go ahead. - Do you have an option or?

[01:16 - 01:21] - Offshading? - Let me see, it's got exclamation parts. - Maybe I'm dull.

[01:22 - 01:29] - How about now? - Let me just see if I'm sure my test out for it. - I'm just doing my shoot.

[01:30 - 01:35] - Offshading's issue. - Let me drop, I'll join right back. - Give me both.

[01:36 - 01:41] - Okay, so I've got to restart. - Yeah, hey James. - Hey, Mr. Condepin.

[01:42 - 01:44] - I'm doing English. How are you? - Yeah, good, thanks.

[01:45 - 01:56] - Do you guys had any chance of going over to their notebooks, mini projects or anything? - I've had some, it's been a busy week at work, but so I've got to get caught up with the weekend.

[01:57 - 02:00] - Nice, okay, nice. How about you, Drew? - Yes, same for me.

[02:01 - 02:10] I have to get started on that mini projects. - Yeah, just bring any errors or anything you have, like in our project, like group coaching calls.

[02:11 - 02:30] And this call is basically to help you solve throughout those errors if you are having any troubles doing any coding or it can be your own project. - Okay, so the session, the one later today are both same.

[02:31 - 02:52] - Yes, basically trying to help everyone go through the codes not just for basically it started with projects, like your own projects, but now like I'm more on to everything. Where you can also bring like any troubles or any errors for coding exercises or mini projects or any.

[02:53 - 02:59] - Right, I'm letting it run. It's got to download libraries for that cell and stuff, so give it a second. - Cool.

[03:00 - 03:03] And what's the copy? Is it the tip?

[03:04 - 03:08] Because I saw the image from it. - Or one exercise, 4.1, the clip matching.

[03:09 - 03:12] That's just one of the examples. - Yes.

[03:13 - 03:20] - For me, again, it was any time you provided the URL like that had uploaded wikipedia.org in it. It never pulled the file down.

[03:21 - 03:28] It would give me a message like, you make sure the URL exists or not. So I just assumed it was like a bot that was like stopping it.

[03:29 - 03:38] - Or maybe they choose the URL works in my browser and I can open up. - Yeah, I see the images working right now.

[03:39 - 03:48] - It's just when I run the program, it acts like it can't get to it. And then I was asking Gemini for help and so I'll make sure the URL was really valid.

[03:49 - 03:53] When I went and found another image, it worked fine. It's not a huge deal.

[03:54 - 04:01] I just wanted to make everybody aware that I was having trouble when I tried to run the exercise. - I see.

[04:02 - 04:09] - But anyway, I'm letting that run for a second. Let me see. 'Cause I wanted to ask you another question.

[04:10 - 04:17] - Yes, if I could and I'm going to share my screen. - Yes, go ahead. - Now I can do this, right?

[04:18 - 04:23] Can you see? - Yes.

[04:24 - 04:29] - So coming through? - Yes. - So I'm trying to just do some basic prompt testing.

[04:30 - 04:38] Talking to my customer. They took a look at the data I produced and one of the comments they came back and said, "Hey, I gave you a product description for a Nike product.

[04:39 - 04:49] The people that do this for me manually, they came back with about 10 features you had for." I was like, "Okay, maybe I wasn't specific enough for my system prompt."

[04:50 - 04:54] It was a nice example. So what I wanted to show you was, "I have my prompt, right?"

[04:55 - 04:59] And let me make this a little bigger. So it's somewhat legible.

[05:00 - 05:09] We were a senior merchandising manager at a leading retailer who was responsible for enriching product content or display about a product on the website. You respond to JSON format only.

[05:10 - 05:16] And then there's a whole list of things they're supposed to be careful of, et cetera. And then I tried to be really explicit.

[05:17 - 05:28] I was like, "Create 10 good feature bullets per product based on the content of the product description." And another feedback he had was like, "Make sure and use proprietary brand names, technologies, et cetera."

[05:29 - 05:36] So then I got into showing just a basic JSON format features with an array. And then my examples.

[05:37 - 05:40] So this one was quite long. It's both of them are.

[05:41 - 05:51] It's the full description about a Nike product with a lot of data in it. And then I actually took the features they had on their website for this particular product 'cause they're about 10.

[05:52 - 05:58] I gave it one more example with a different one. And again, their website only had five features for this one.

[05:59 - 06:07] So I don't know, maybe that's somewhat informing what the model's doing. And then as I basically said, using this product description, create three.

[06:08 - 06:10] Oh, this is probably my error down here. Let me see.

[06:11 - 06:23] I'll put 10 style attributes and see what happens because what was happening was I was running it and it kept getting like between four and six depending on the model I was using. So let's try it.

[06:24 - 06:27] I switched to commercial models to see if that was better. And then this is.

[06:28 - 06:32] - Which is the model that I was using? Is it the same one that we talked about? - I was using deep-seak.

[06:33 - 06:39] And then it was coming up with only four and I switched to GPT-5 and it was giving me six. So let's try both.

[06:40 - 06:41] We can see what it is. - Yes.

[06:42 - 06:46] - It outputs. Maybe it's gonna come up with more this time. Yeah, so it was my prompt.

[06:47 - 06:57] - Yes. Just a bit of the prompt. And then it's always, the reason is like, what we do with the prompt is we optimize the prompt, basically with another LLM.

[06:58 - 07:07] And then maybe that is the reason like it produces those error and then it carries out like with other things. - Yeah.

[07:08 - 07:10] Let's see when it'll come up three, four, five, six. Yeah.

[07:11 - 07:15] Yeah. So I'll try with deep-seak and see what happens.

[07:16 - 07:18] Well, maybe not as many, but... - How many?

[07:19 - 07:22] - Nine. - Nine. - No, they got 10, didn't it?

[07:23 - 07:24] - Yeah. - Yeah. So it nailed it.

[07:25 - 07:27] I need to read them. - Now, yes.

[07:28 - 07:34] So basically we got the 10. But if you want to be consistent, what is the temperature we are using?

[07:35 - 07:44] - So I don't have a temperature set. I'm using, that's actually one of my questions. I asked Alvin that earlier and he didn't, he wouldn't really want to be able to answer it, but this is my form hat.

[07:45 - 07:47] And I'm using open router. - Yes.

[07:48 - 08:00] - Do you know how I put the temperature in? - Let me see if I can... 'Cause I was wondering if I'd changed it, would it be more willing to write more?

[08:01 - 08:12] - Basically, the temperature is something with the prompt. So it's a brute force approach. Like you will have to find the right temperature, but it's between staying consistent with the prompt.

[08:13 - 08:24] But at the same time, you also want the creativity, says that it's not too consistent. Or else it's gonna generate like similar description for various different products.

[08:25 - 08:31] - Yeah. - It really needs to be probably a little more strict, but with some flexibility. - Yeah, maybe like a start.

[08:32 - 08:51] For your use case, I would come and start with 0.7, and then increase it until you have consistent response for all of the different products. So for example, let's say if you are teaching it with 10 products, then maybe you have to check the temperature with those 10 products.

[08:52 - 09:19] - Yeah. - And basically, you can always choose the higher number of, depends on the application, but I would recommend on your use case, always use the higher number of temperature such that it has the variation on-- - Do you think all I would need to do is just put another like comment here and do a, I don't know if it will make a temperature. Yeah, so I guess it does.

[09:20 - 09:26] - So you said 0.7? - Yes. - Okay, correct me.

[09:27 - 09:50] - So anyone wondering what's going on? What we are doing in here is, I think it's really-- - Michael, I'm curious, like you mentioned with Alvin earlier, have you done anything with an intermediate step or like giving it like categories of feature descriptions? Like some feature descriptions are about fabric and some are about fit or-- - I think there's any way of, yeah, if you give it buckets, try and fill up these buckets until we get time.

[09:51 - 10:06] - There's two pieces, that's a really good point. You call out, let me see if I can show you just in a browser, what some of my challenge is. 'Cause a lot of it depends on the data and the data is very inconsistent from the brands that provide it.

[10:07 - 10:26] - You gotta find my spreadsheet to show you here, right? Go back, share this so you can see, we'll share the best on. - You guys let me know if this is too small, like I certainly make it a little bigger, make it easier to read.

[10:27 - 10:46] So this is an example, what generally happens, they have the list of products they wanna build features for. And so you get a name, they're internal identifier and something that maps to the vendor and then I get a second spreadsheet or a sheet that's actually from Nike.

[10:47 - 10:56] And so in here, it'll map to the style code. So I can programmatically do a look up and find it and then I get this copy.

[10:57 - 11:08] And the copy's very long, it's not always great, but it's got some bullets. And so the basic task is take this and create, call it eight to 10 bullets out of it.

[11:09 - 11:16] And ultimately, like what this kind of looks like on their website, it's not really an example. - And what's this far in like?

[11:17 - 11:25] - Like a metal one. - Do you remember in the name of the shoe I was looking at this morning? - Yeah.

[11:26 - 11:36] - I think that's an example of one. - So on the page, have a description, you have an even the product. And then down here, this is the feature list.

[11:37 - 11:45] So this was step one to generate for them. But like the other side of it is that they also want text specs and these are much more constrained.

[11:46 - 11:58] So like for this bucket, but for flickwear, they always want to have an activity, the cushioning, the midsole, the height, sometimes like a drop, whether it's some millimeters or 10 millimeters. And then there's a set of defined values.

[11:59 - 12:16] So I don't know, maybe I can guide it like you were saying for the features, but they tend to be a little more free form that are built off the description where these are definitely like constrained by a domain set of values at the end of the day, if that makes sense. - And it's interesting too, 'cause this is just one example.

[12:17 - 12:27] They have given me examples for Yeti. So if I were to open up Yeti, their inputs better, in my opinion, than 90.

[12:28 - 12:36] They actually have features already called out. So it's more about rewriting them and seeing if you can come up with any additional ones at the end of the day.

[12:37 - 12:49] And so it's interesting, it just, it's all varies. And I'm trying to figure out how to create something that's somewhat resilient to look at the inputs and know whether things like this exist or not.

[12:50 - 12:53] They don't use the description. I've got like scraping involved.

[12:54 - 13:07] So I can go look at the brain's website and see what they've done, 'cause the brain's website was valid. And so what I've been doing to show them my work is that I give them like three rows per product.

[13:08 - 13:15] I show if there was anything in the vendor's spreadsheet. So like in the case for Nike, there was nothing, but I had a description.

[13:16 - 13:22] I go look at the website, so I'm using that fire crawl tool. And I get to a search, and then hopefully I get a match.

[13:23 - 13:30] I don't always get a match, but it tries to extract some features, but you can tell these aren't great. They're not really about the product.

[13:31 - 13:41] And then I do my AI generated version. And this is where he's saying this morning, I notice you were quite getting the same number of features that my offshore team is doing.

[13:42 - 14:05] And he asked me to make sure that I was really emphasizing like the tech, like drive it, if they have other things they call out that are very specific to the, to the brain's fabrics and stuff. - Yeah, so that should be in the product itself, where maybe you can have a bullet point of product description properties, and then or maybe emphasize these properties.

[14:06 - 14:12] And then one, and then some bullet points for the tech, then the fabric material. - No. - Yeah.

[14:13 - 14:27] - So I should probably enhance my prompt a little bit to do that, 'cause I tried to say, remember use proprietary brand names and technologies were applicable, but I probably need to be more explicit, maybe in the examples I give to. - Yes, exactly.

[14:28 - 14:57] Like the examples, so the examples are basically taken with the perspective of how it should generate all the writing style, but the emphasis is that you mentioned, so we have to explicitly mention the emphasis in the prompt itself, where basically you have this list of prompt award duplicating any product specification when you detail list. So there we have to add a prompt on emphasis.

[14:58 - 15:13] And then emphasis can be like a property of what we are emphasizing. So the property here is the tech that the clothes are using, the fabrics that the clothes are using, and then all the properties that they are mentioning.

[15:14 - 15:21] So maybe like you can take the precise emphasis is parameters from them. - Maybe to make sure I understand what you're saying.

[15:22 - 16:04] So like before sending any of this off, like maybe there's, is it like research I need to do on the brand or the, and to say these are the, I don't know, to whatever method, pull out the extract, the tech features, and then include them in the generic prompt, or just say make sure, I'm trying to figure out how to give it the right guidance to know like what is the tech that's being called out by these brands. - I wonder if that would almost be worth doing if you did your prompt like a sequential prompt and got it to identify brand terms as a first step, for how do you, when you get that chunk of text, how do you segment it?

[16:05 - 16:26] If there's a, I mean, there's like a way where you can, there's something I've been struggling with as well where I'm working with something that it's like engineering documents, and then it comes with a technical glossary, and I'm trying to get it to recognize like the glossary terms as like almost like anchor embeddings almost, like these are more important than the rest. - Yes.

[16:27 - 16:54] - You know, basically that should be in the prompt itself where you have, we have to emphasize, you have to emphasize that specific anchoring and those are the leading anchoring. - So I guess to be clear on your advice here, the benefit is that I guess in addition to showing it in an example, like when I say something like, and you know, remember to use proprietary brand names like X, X and X?

[16:55 - 17:13] - Yes. - You know, and then technology is like, technology is like X, X and X, where the X is the real variable in your domain. So it should not be like that, it should not be X, but it should be the variable that you are using.

[17:14 - 17:19] For example, can you go to the product EI generated Excel behind your code? - Yeah.

[17:20 - 17:27] - So it can be, it can be the drive, it can be an example, right? - Yes.

[17:28 - 17:39] - Drive it, and then for another thing, it can be something else. So maybe try to give examples of what technology mean in your domain.

[17:40 - 17:59] - Okay. - Or you have to find another word which maps exactly towards those words. So for example, we are calling it technology, but it might be like some other words, so maybe you can like probably research like brand technology or something like that.

[18:00 - 18:16] - Yeah, yes, exactly. - So maybe to go back to James' point though, like I was thinking about what he said, this company sells lots of different stuff from Yeti coolers to tennis rackets to rifles, you name it, like golf clubs, all over the map. But does there need to be a step?

[18:17 - 18:27] 'Cause usually they do this only for a brand and really a subset of a brand's products like when they do the assignment. Let's work on Nike shoes or let's work on Patagonia jackets, whatever it is.

[18:28 - 18:47] Is it worth having an agent go? Okay, go figure out like what the tech terms are, the glossary to use James' term for this products or so that you can then inject them into the prompt specifically when you're doing Nike or-- - Yeah, that is more preferable.

[18:48 - 19:06] - So it's better than just showing it. Like I was thinking the Pete's example yesterday we had were Zal kept saying, you don't have to tell it what basil is, it will figure out what basil is, but-- - Yes. - In this case, it sounds like it's almost better to tell it what the universe is than let it have to figure out what the universe is.

[19:07 - 19:19] - Yes, basically what the architecture that you mentioned is maybe like a week to architecture of what you are implementing. Basically, you will need like further enhancement as you complete life, you have complete life.

[19:20 - 19:33] - 'Cause I was saying anything about how to do that. I guess one, I could take every description in a year and try to extract what I think are terms that seem like brain technology. Maybe there's a way to look at the website and find something.

[19:34 - 19:52] Sometimes they get PDFs that are really big and lengthy that might have an explanation out of the tack in them where stuff could be extracted, but what I wouldn't want it to do is just to incorrectly assign the tack to a product in a creative process and be like, "Yeah, this uses dry fit and no, it doesn't." - Yes.

[19:53 - 20:00] - Then it's a bigger deal to them. - No, it should be like with the product description itself. - Yeah, okay.

[20:01 - 20:11] - You guys, if you directly link it with the website, like it can have like multiple things. - You know, yeah, I was, and again, I don't want to get ahead of myself.

[20:12 - 20:19] There's this thing about this, to be applicable to more than just one customer who's given me this information. - Yes.

[20:20 - 20:34] - But over time, I could imagine that you build up this database of product information, right? Like you've collectively got to understand, create a graph database, whatever it is that shows the association of this tack to these products.

[20:35 - 20:41] So I don't know, that's the other side of it. Maybe eventually this gets augmented with some of that data.

[20:42 - 20:57] - Yes, at some point, you will basically need data blocks or for the tech data. Basically, also it depends, for example, so right now what we are implementing is something similar, or the keywords.

[20:58 - 21:10] So it's basically the keywords engineering. And we are doing something similar for the community where what we are trying to achieve over here is you give a prompt in the notification.

[21:11 - 21:27] And then it only gives you the notification diverged towards the prompt. For example, I'm only interested in the notification that our mini projects or that are just AI tags.

[21:28 - 21:39] And I'm not interested in foundational AI concepts. Then basically the notification that you will get are gonna be just AI tags and not the notification with foundational concepts.

[21:40 - 22:00] Like it's basically similar of what you are seeing where what we are doing right now is we have the turbo buffer that ingest all these threads from the community and all the materials. And we are trying to find a similarity from your prompt to fetch those threads.

[22:01 - 22:08] And then if there is a new thread, then it gives you the notification if it falls in the similarity range. - And see you.

[22:09 - 22:18] - Okay. So James, you're doing engineering related stuff. Like, no, I think you're on mute.

[22:19 - 22:32] - I've experimented with it a little bit. Just we're trying to get a hold of our, get a handle on RAG. I've been using some of the like engineering manuals that NASA publishes just since it's all like open source and really structured.

[22:33 - 22:40] Yeah, it's still at the level of just trying to get it into embeddings and tokens and being able to do the lookup of it. But, which library are you using to do that?

[22:41 - 22:49] - Right now I've just been using like light chain and like the standard like open AI transformers. Sorry, I didn't necessarily, what are you trying to achieve?

[22:50 - 23:07] - That was just a working on like a RAG application of being able to query like a specialist knowledge set. And then I think my goal after that is since it has a pretty defined process or input, destruction output and engineering documents and specs and gates and that kind of thing.

[23:08 - 23:25] I'm starting with the lookup, like the RAG of tell me information about this process or rewrite this using the language from this document and then working up to, okay, I've described the problem space, put out engineering requirements for it and limitation steps. - I see.

[23:26 - 23:39] - So basically the first step. So what's the first phase of the project? So what was the first phase that you are trying to achieve is ingesting engineering document and then querying it through?

[23:40 - 23:44] Did you started like evaluation? - I really haven't made it to the evaluation point.

[23:45 - 23:49] All right, now I'm really just at the ingesting point. So yeah, pretty small steps.

[23:50 - 24:03] - I see. So maybe the most helpful thing that you can find is go over to the onboarding. So that is the usual, so the usual approach that people take for the RAG is not as helpful.

[24:04 - 24:16] The reason for that is you won't be able to maximize your reasons. So what you can do, you can go to tutorials on boarding, debugging and environment catalog.

[24:17 - 24:27] And then in there, you can see like common mistakes to avoid. And then just skin, like skin through or read through the EID debugging and Ewell centric RAG.

[24:28 - 24:35] And Ewell centric, yes. - So we are like, we are going to cover that in the next lecture itself, but they, okay.

[24:36 - 24:39] So let me show you the approach such that it's more helpful. - Oh, sorry, sorry.

[24:40 - 24:42] Interruptible, I accept a job for another meeting. - Okay, sounds.

[24:43 - 24:45] - So I'll catch up with you in the next section again. - Yes. - Okay.

[24:46 - 24:52] - Thank you though. - I have just a question. When do you bring evaluation into the frame?

[24:53 - 24:57] I mean, to the lifecycle, right? We do say store the document and retrieve it.

[24:58 - 25:05] So Ewell, Ewellish, sorry, Ewell should be part of which lifecycle of the product. And how do we keep it right?

[25:06 - 25:16] What are the rubrics that you recommend to keep in track? Because I've used it deep and they give some metrics that comes with the default, like contextual relevance and then answer faithfulness.

[25:17 - 25:20] So Raga metrics, right? - Those are like two different Ewells in there.

[25:21 - 25:26] Basically one Ewell. So the one that you mentioned is the generation Ewells.

[25:27 - 25:38] And the other one, the more important one is basically the retrieval Ewell. So let me show you the things on how I usually approach the any rec problem.

[25:39 - 25:46] So this is gonna be available for Ewell after the meeting project. But yes, can everyone see this screen?

[25:47 - 25:54] - Yeah. - So this is how I evaluate the Raga. I don't just go for two or three experiments.

[25:55 - 26:02] Here I have 24 experiments. And then basically the reason I'm having the precision lower over here is because of at the rate five.

[26:03 - 26:11] But when you go to the evaluation, like deep, I think it's like in here somewhere. If you go in there, then you can find like real precision at the rate one.

[26:12 - 26:20] But what I'm doing over here is, so these are all the retrieval evaluation. And we are gonna cover it in the react lecture, like tomorrow's lecture.

[26:21 - 26:33] But basically the important one over here is recall at the rate one, three and five. Similarly, precision at the rate one, three and five.

[26:34 - 26:54] And then MRR is basically, you can think of MRR as mean result of mean retrieval result, yeah. Basically, you have to find the right balance of all of these things because you will never have any retrieval where you will always have your preferred option on the top.

[26:55 - 27:05] And then you also have MAP and ND CG. And one of the common mistakes that people usually do is they do count the time, the average time.

[27:06 - 27:20] So always, yeah, always count the average time too with those experiments. So how I usually start the experiment for the RAG is basically, I have these different kinds of, I have these different kinds of chunking parameters.

[27:21 - 27:31] So 25650, which is basically 256 characters and 50 overlap. And then similarly, 51200, similarly different types of chunking parameters.

[27:32 - 27:42] And then I also have different types of, I also have different types of embedding models, which is used from one EI API. You can use like hugging phase API.

[27:43 - 28:07] But what I would recommend is using one EI API because usually you will get the best performance using any of those three models, depending on what you are trying to achieve. And then for vector DB, like vector DB over here, the preference of vector DB over here is solely for the time, like retrieval time, because as you have like your product on scale, time is gonna increase exponential.

[28:08 - 28:23] So right now I'm using like turbo buffer and the local one, but you can basically integrate more vector demos if you want. And then different types of retrieval methods, which is basically one of the retrieval methods in BM25.

[28:24 - 28:32] There are other retrieval methods that you can use. And then there is like vector search and then the hybrid, which is basically using two retrieval methods at the same time.

[28:33 - 28:50] The reranking do not worry about the reranking right now because it's like a step after you get like those 24 reasons. And then you also have to generate synthetic data for the web in order to evaluate the system.

[28:51 - 28:59] So what you do is basically you first ingest the knowledge with all of these options. And basically you will have 24 different options.

[29:00 - 29:20] And then you generate equations with all of these different DBs, from all of these different DBs with their chunk ID. Now, for example, option E will have one chunk and based on that chunk, you generate a variation of 10 questions.

[29:21 - 29:43] Now those 10 questions should have the, should have a variety of your domain. So for example, if you are doing reg application for customer base or customer support, then it should have variation as angry customer, humble customer because these different customer also now will ask the same thing, but in different way.

[29:44 - 30:08] For example, angry customer can ask the same thing, like how to contact customer service in different, in several different ways. So you need like variation of these questions for the same chunk and then basically add a goal, add a chunk ID, add a chunk ID, and then you will have a complete data set for that specific vector DB.

[30:09 - 30:17] Similarly, do that for all. Now the thing over here is, another thing is people try to avoid generating questions between the overlaps.

[30:18 - 30:36] You have to always generate questions. You have to generate more questions between those overlaps because that's where the reg gets confused and then basically you use those data set to retrieve the chunk IDs with the help of a retriever and then get all this course.

[30:37 - 30:59] These are all the retriever schools with the time. For example, over here, I would prefer not to go for the highest one because it has 0.47 time, but also maybe I can go for this one and then I can use like reranking on top of this to push the scores.

[31:00 - 31:09] So that is basically, that is basically the approach that I usually use. Do you have any question with the approach?

[31:10 - 31:21] - So reranking mostly we use it on the hybrid search, right? Or I just wanted to get more understanding about the reranking. So when you said about the hybrid search, we use both the keyword search and also the vector search.

[31:22 - 31:30] Then you rerank or the documents, which is most relevant. And then you display that whatever the relevant top five documents are what.

[31:31 - 31:37] That's why I use the rerank. Basically rerank requires training and it will be like on the Rev node.

[31:38 - 32:07] But okay, so the basic way of understanding the reranking is for example, this is the score, but I only find out things that has the chunk ID in top five, but I want that fifth or third or some other chunk ID is to be on the top one or two. I have to use reranking and then I have to train the reranker in that way in order for me to retrieve all those chunks in one or two.

[32:08 - 32:25] - I see. - Yes, correct. - So basically reranking is like prioritizing the chunks within top give result, but like pushing that gold chunk over the top one or two.

[32:26 - 32:29] - Two ranks, yeah. - Always you get at the top. - Yes, exactly.

[32:30 - 32:38] - Okay. - But is that normally preferred in the hybrid search or in the normal, in the vector search? Also we just try to rerank.

[32:39 - 32:48] - You can basically use like reranking anywhere. - Anywhere? - One of the best reranking EPI is cohere reranking.

[32:49 - 32:55] So this is the one that most of the people uses for the production. Maybe let me share the links as that.

[32:56 - 33:03] - Sure. - And there are also open source rerankers on the hooking phase. So this is the cohere reranking, yes.

[33:04 - 33:16] This is the production EPI that people usually use for the reranking stuff. But the downside of reranking is once you train the reranking, you won't have those basic results.

[33:17 - 33:32] Sometimes you might, yeah, you might deplete that basic result. So now you have to find a way to make sure you don't deplete those basic results, but also at the same time you emphasize the maybe like last chunk IDs on the phone.

[33:33 - 33:57] - Correct. - This project coating is for our end project coating, right? - Yeah, the problem of the call is basically if you have any problem with your project, if you have any problem with mini project or if you have any coding errors, you can basically share it right now and then I can walk through and guide you through from that hand.

[33:58 - 34:09] - Yeah, sure. I think I haven't what you call categorically, put it in paper, the final project idea, but I had some work done that idea from the backwards. I can share that but I will put up an architecture.

[34:10 - 34:22] The idea that I'm working towards is very similar to course. I want to build an ID for only for the data engineering, from the prompt to analytics or to building the data pipelines I wanted to use because that space I've been working.

[34:23 - 34:26] So I have certain understanding. So I build the crawlers and build the pipelines.

[34:27 - 34:37] I started with us before joining here, I started with a simple agent approach, building the crawlers. I mean, using, built a product on heterogeneous data migrations by using agent system.

[34:38 - 34:51] So I just wanted to focus that. And then my goal was to use Ker's ID just to get the prompts and then use the agent approach along with our RAG systems to build that pipelines and to have that contextual editing of the product project.

[34:52 - 35:08] So that I do have an architecture sketched in my paper, but I will just share it with you in the email or in the community and then I will get your thoughts from there. But I've built part of the agent with me and knowledge I have at this point, but I will, I totally improve it.

[35:09 - 35:16] But so that's the product I'm trying to achieve. - It pretty bad love to see more about what you're doing.

[35:17 - 35:22] It sounds interesting. There's never a time when you want to talk, share, collaborate and be interested.

[35:23 - 35:27] Sure, Michael, we'll do that. - Yes, Michael is working in similar problem, right, Michael?

[35:28 - 35:49] - Yeah, I mean, my goals maybe you saw earlier is just I'm dealing with like enrichment of product data and mainly targeted towards retailers and distributors that sell a lot of other people's stuff. And they go through this laborious product enrichment phase and they're dealing with a lot of inputs that vary mostly in spreadsheets, sometimes PDFs.

[35:50 - 36:04] And I'm trying to make content unique for publishing on the website and I'm trying to think about to each customer is going to have different informed formats for backend systems, right? Because someone uses SAP, some use Oracle retail or whatever.

[36:05 - 36:19] Yeah, I feel like I almost at some point going to have to have some kind of transformation or code generation stuff to take my master model product data and move it into the formats they need for import or facilitate the imports. That's what I'm trying to do.

[36:20 - 36:34] - Great, yep, sure, Michael, I'll collaborate a very similar problem, like I think I mean, that I will share the architecture here and then you can recommend from the agent perspective how I can move forward. - Yes.

[36:35 - 36:40] - And any of the techniques that I can use. - Yes. - Pretty poor you look at it.

[36:41 - 36:46] - I'm looking at it in Toronto. - Okay. - Really used to, I lived there in Houston.

[36:47 - 36:50] Yeah, so I just been recently more here. It's good, it's good.

[36:51 - 37:01] What I was gonna show you, I think I can show you the issues. Dippin, this is what I see in that notebook when I use an image from Wikipedia.

[37:02 - 37:21] - Yes, I think, so I'm trying to correct the error. Basically, originally I had the WGAT where it downloads the image on your one drive and that you use that image instead of using the pill. But the editor might have changed the entire code.

[37:22 - 37:28] - Yeah. - Yeah. - Again, if I switch the image down it works to something that's Wikipedia.

[37:29 - 37:47] - Yes, I think that's a crawler protection wire because the pill is very old library. So the solution around that is making the streaming crew such that the image loads in the back and then basically can take the image.

[37:48 - 38:03] - Yeah. - Oh, yeah, that's something I also have the questions around it because I worked on the scraper but I think it's very internal to the enterprise. So it's easy for us to overwrite that and then able to do the scraping.

[38:04 - 38:10] But I know most of the sites have been bought, controlled, protected. What other libraries do you recommend depending on overcoming these challenges?

[38:11 - 38:18] If you haven't scraped the public source in a measurable, I mean, in a safer format, right? But we don't flag them that does.

[38:19 - 38:39] - Yes. So as I recommended earlier to Michael, this is one of the very full episodes that I use for scraping the web, scraping any website which is basically fire crawl.dev. And then there are another crawlers that I will also disclose which is basically, I think, agent QL.

[38:40 - 38:57] It's like a more advanced version of crawling and that is one of the other crawler that you can use. But the agent QL is not user friendly while the entire crawlers more user friendly.

[38:58 - 39:14] - Yeah, I've been using fire crawl a lot and it is really good. It has a basic attempt and if it doesn't navigate that defense, it has like something called stealth mode and it charges you a little more but from my experience right now, the basic mode.

[39:15 - 39:33] I haven't seen any kind of excessive modes and it's nice because you can also set like a JSON schema and it's smart enough to look at the page and infer like what you're asking for. So like in my case, I said, I want to name a description, features, specification, and the form of an array.

[39:34 - 39:38] And it does its best and it's not bad. It's, I think it's good at the crawlside.

[39:39 - 39:54] - Yes. And there is another open source. If you want to have your own, so you can have two options for having your own side web crawler, which is basically fire crawl is like an open source program. But there is another one which is this one.

[39:55 - 40:05] I forgot to mention this earlier, Michael. This is like crawl for EI and this was like one of my first before, like referring to fire crawl.

[40:06 - 40:26] This was one of my previous preference where if you are targeting to host your own crawling agent, then this is the one, instead of using like API, but if you mix like everything easier. And then once you have a project ready and then basically you can start enhancing it with your own side, on side.

[40:27 - 40:31] - Sorry, Paul, you are. - Yeah, sorry interrupt. I will log off like I just got paid for P1.

[40:32 - 40:33] This week, I'm an on call. So yeah, sorry for that.

[40:34 - 40:35] Yeah, I'll try. - That's good.

[40:36 - 40:40] - Thank you. - I would like to have this conversation regarding me this conversation for you. - Yes, sounds good, sounds good.

[40:41 - 40:45] - Thank you, Michael. And thank you. - Yes, Michael.

[40:46 - 41:01] - Does crawl EI do the bot avoidance or you have to build that in and start changing headers and stuff like that? - I think, I think it does at some extent, but if it has like advanced capture, it might be up.

[41:02 - 41:05] - Yeah. Okay. Interesting.

[41:06 - 41:07] All right. This was good.

[41:08 - 41:09] Thank you, Deppin, as always. Appreciate it.

[41:10 - 41:18] - Yeah, just, I will add in comment. I will add the comment in the search line. I believe I'm adding one.