A Glimpse into our Group Coaching Calls 5

[00:00 - 00:03] Ducky TL started to implement it. - Yes.

[00:04 - 00:14] - And I think it's much better than the jumping approach 'cause now I have a structure and looking at all those different PDFs for running shoes. It actually is like doing a classification.

[00:15 - 00:23] So it will say whether it's a product detail page and the PDF or if it's something else like table of contents or whatever. - Yes.

[00:24 - 00:31] - Which is helpful as well. So I think that's gonna work better. - Trying to put it into that full rag pipeline that I did for the project.

[00:32 - 00:37] So I'm in the throes of that. I'm putting it in a turbo puffer, trying to do the structured eval now.

[00:38 - 00:43] - Yes. - But I think I'm heading in the right direction. - How are you gonna use the Ducky TL?

[00:44 - 00:53] Are you implementing this EPI where you use your own EPI call the Ducky TLR? Like how are you gonna implement?

[00:54 - 01:01] Have you thought it like in terms of production? - So what I was thinking, 'cause I think what the end user's gonna do, they're gonna have access to these portals from the brand.

[01:02 - 01:06] So they'll download these files. They'll probably stick them in a Google Drive somewhere.

[01:07 - 01:24] So what I was thinking since they'll be big, either I can give them an upload in the user interface or I can just say, hey, give me access to your folder and I'll go read the files. And then what I was gonna do is use the Ducky TL library and just parse through it.

[01:25 - 01:29] And then right now it's spitting out, I guess JSON files. - Yes.

[01:30 - 01:35] - And then I would have another step that would take those and build the index with them. And then it would be in there.

[01:36 - 01:43] And then I have to, so I was thinking it's kind of like building like this product warehouse. - Yes.

[01:44 - 01:49] - More information. And then they started enrichment stones. Say, okay, now I wanna do these shoes and it looks in there to find it.

[01:50 - 01:58] - Yes. No, I think you misunderstood. What I was trying to ask was, so right now we have like Ducky TL and everything working in the pipeline.

[01:59 - 02:12] But when it goes to production, you might have to change some things. So for example, so let me, so for example, so we have three actions in each of the chat now and it's like up and working.

[02:13 - 02:24] And these are the three things that people always wanted while they were watching the videos. So what I did was like data collection from that user, which is you and everyone else.

[02:25 - 02:38] They wanted to summarize, but this is not a simple summary, but it's more of summarizing content with prompt engineering technique, which is basically the time steps. So if you go over in here, it's very easy.

[02:39 - 02:44] So I wanna go with bootstrapping concept. You just click on here and then it's gonna start with bootstrapping.

[02:45 - 02:59] So these are some things that are not in the pipeline, but as a API. So I have hosted this in the local container where it is serving as API.

[03:00 - 03:04] So this is what it's calling. So this is what it's calling in the back.

[03:05 - 03:10] - Okay. Let me make sure I understand kind of what you're saying. So you've ingested a bunch of stuff in the Turbo Puffer.

[03:11 - 03:27] Now you've got an interface over here at the chat where if you want to get access to that original content, you call it, that's your API, right? It's the API back in the content that sits in either the original document or Turbo Puffer or both.

[03:28 - 03:41] - Boot. So what happens in the backend is when I click in here, it calls an API where it retrieves the original data from lesson ID. And then, so this is happening on the API.

[03:42 - 03:56] And then the API is gonna, so the reason I'm calling summarize, then the API is gonna use that summarize from the engineering technique that I have in the backend in the API itself. And then it's gonna produce an output for the end user on here.

[03:57 - 04:15] - All right. So I'm trying to think about my case 'cause if I've ingested these documents, so I've created a record now in Turbo Puffer that's, it's not chunks, it's like description features, this, some vector embeddings. - So that's why I wanted to discuss such that I can give you more, okay.

[04:16 - 04:30] So there are two ways of doing things. Either you can use trigger.du, which is basically something that I'm using for our SEO bot, AI, web application, or you can use, or you can go with the architecture of API.

[04:31 - 04:40] So in here, in here, the architecture is basically, so these are Python files that are on the backend. And so it's gonna fetch like all the things.

[04:41 - 04:51] So when I click on generate headline, if you remember, in here, when I click on generate headline, it's gonna execute, it's gonna start the machine first. - And it'll start a job, right?

[04:52 - 05:07] - Yeah. And then it's gonna execute the file, whether it's generate headline, generate article, or any of it. Or you can go with EPI's that are active, and then you can basically use API.

[05:08 - 05:18] But these are only for small jobs. - Yeah, I'm still trying to make sure I have, I have your point like in my new case, like I think I, 'cause I have all this data now sitting in turbo puffer.

[05:19 - 05:28] - Yes. - And I have the original document too. When we were talking yesterday, I was thinking about like, eventually when I say enriched the product, right?

[05:29 - 05:34] I'm a user and I say, I'm gonna look at this turbo puffer database and say, what do I have for this product? Oh, I've got stuff.

[05:35 - 05:43] Oh, maybe I don't have everything, maybe I do a web search. So now I've got the full corpus, now I ask the LLN to rewrite, then the user's gonna see it.

[05:44 - 05:56] And they might wanna go description feature, go look at each one of them individually and thumbs up or thumbs down the eval. And so there they might be like, okay, what's the source for this?

[05:57 - 06:02] Click on it. Then I think I could have an API that would be like, okay, go retrieve.

[06:03 - 06:10] - Yes. - Maybe like in your case, you're showing the video. In my case, it's a PDF that goes to that page - Yes.

[06:11 - 06:20] - Shows where it came from. - Yes. So I think the web search and all the things are gonna be on trigger.du or jobs, basically.

[06:21 - 06:32] - Yeah. What is trigger.dev again? - It's an abstract level of platform so that you don't have to go on EWS and you don't have to like trigger.du is more of connecting.

[06:33 - 06:44] - Or fire is a workload off if you do something. - Connecting the front end, back end and the python screen. Basically, this is very easy to integrate on web application.

[06:45 - 06:57] It's what happened, like you would have to configure it with the, I forgot the machines that we are using. - Is the trigger your background jobs?

[06:58 - 07:00] That's how you run your background jobs? - Yes.

[07:01 - 07:17] So when a user clicks on here, what happens is it's gonna execute a JavaScript and that JavaScript is gonna execute the python file. Well, I think you might, you will have to configure it like whenever you sign up.

[07:18 - 07:37] And then what we are using is digital ocean and then it's gonna use the digital ocean machine or you can basically just use their machine, which is, and also you have to make sure, so we did this mistake. You have to make sure that the machines are selected on the right region.

[07:38 - 07:46] The reason for that is like what happened before, like it automatically selected the machine location. And then it was taking longer time to find that machine.

[07:47 - 07:51] - Okay. - But shorter time on processing. - Okay.

[07:52 - 07:53] This is trigger.dev. This is an SC.

[07:54 - 07:56] This isn't your SEO buy thing. - Yeah.

[07:57 - 08:00] - This is where you can see the jobs, the queues. I got it. - Yes.

[08:01 - 08:05] And then also it's not just that. You can basically check the logs too.

[08:06 - 08:21] So for example, let's say if I'm having any, so these are all CLI logs. So whenever I'm building an EI application, I have the tendency of going thorough and logging each thing such that I know what's the error, if there is any edge cases or something.

[08:22 - 08:31] So you are gonna have this console. So if you have console logs, when you are building right now, you are building the EI application, you are gonna have these console logs too.

[08:32 - 08:42] - Yeah. - What I did right now, so I created like a queue system. I used a technology called NATS, which is like cop goves in the same way.

[08:43 - 08:51] So what I, when somebody creates a product, I actually put an event on the queue. And then I have a worker that just listens on that queue.

[08:52 - 08:57] So if it hears the event at once, which is product create and it starts the job. - Yes.

[08:58 - 09:04] - And runs in the background and then I integrated a Grafana. So I use the open telemetry logging.

[09:05 - 09:09] - So that's where I've been sticking it. I think this is interesting triggered.dev. I wrote it down 'cause.

[09:10 - 09:27] - So this is more on abstract level. You don't have to go in cloud similar to, so for example, turbo buffer is built on EWS. Like when we are building application, we know what are the functions of EWS, how to do all those things, but it takes too much time.

[09:28 - 09:34] - Yeah. - So this is more on abstract level, which makes things easy on the first pass. - Yeah.

[09:35 - 09:44] - So basically what you're using is like on the pro level. Once you are at certain stage, you can start shifting to more fundamental elements. - It's a good call out.

[09:45 - 09:54] I probably spent too much time last week doing this stuff. But you know, what I realized last week was, 'cause I really want to get this so that a user can start using it.

[09:55 - 10:03] 'Cause I've been the interim person. I sent me the files I do and I sent them the results and I want them to use it. I spent, I showed you the user interface yesterday, but I added stitch for authentication.

[10:04 - 10:11] I wound up having to think about all my entities. I'm storing, they have an org ID on 'em, 'cause it's multi-tenant, right?

[10:12 - 10:21] To make sure the security was working. So it's kind of like one thing after next, and then I wanted to have the tracing, 'cause I know what there's gonna be issues to track down, but I like what you showed me.

[10:22 - 10:23] I wrote it down and I agree. I'll investigate it.

[10:24 - 10:29] - I will. - Yeah. It will mean, it will just make life easy when doing the jobs.

[10:30 - 10:41] Basically what you're doing is like on the very fundamental level or very rule level, where whenever we... So this is basically a cloud engineering job, like what you're doing.

[10:42 - 10:47] - Yeah. - It's just a... - No, I know, it's a good point.

[10:48 - 10:53] Like I need to focus on getting this whole process, right? With ETL, the job running, the results working.

[10:54 - 11:14] Let me ask you one other question, 'cause I was doing my research yesterday after I talked to you. And integrating doc ETL, but there was one other technology that I guess through some of the chat TBT conversations I had, and they were referring to something that Microsoft came out with for OCR.

[11:15 - 11:18] I'm trying to remember that you probably remember the name of it. There've been like three versions of it.

[11:19 - 11:36] - Yes. People from, actually I have never used Microsoft OCR, but people from previous cohort, like they had amazing results. And the end project, like by the end of COVID, they were displaying their insert and they were quite relaying on the technology.

[11:37 - 11:53] - Yeah. What I thought was interesting was they had an example of training it, like on your subject, a expertise. So they were showing like a PD out, you go through, you label it, you send, I don't remember, 20 examples through, 30 examples through.

[11:54 - 12:03] And then the retrieval was supposedly the inference becomes better. And I just, I didn't know, I'm trying to remember the name of it, like I was looking at it yesterday.

[12:04 - 12:11] What was the name of it? It was like, I mean, you could go a little quick.

[12:12 - 12:31] - But if it was training, then doesn't it lose our purpose of the unstructured data? Because we have unstructured data and if we train with specific, specific city, then it loses our purpose of right or is it just random samples?

[12:32 - 12:41] Like, I don't have any knowledge on that, but the results from Microsoft were quite fascinating on in previous school. - Yeah.

[12:42 - 13:00] - They were basically, they were using the same thing. So for example, you had the shoes and all these different PDFs with different structure. And the guy had like different invoice from all over the USA and these local noise were in different structure, but they had the same thing.

[13:01 - 13:20] So for example, in your case, his product name and then some features. And while in his case, it was a vendor's address, it was the customer name and some sort of specific thing, even though the data is like unstructured, but those specific fieldware present in all of those unstructured data.

[13:21 - 13:25] - Got it. - Yeah. - I can't find it, whatever reason, I can't find it now, but I'll send you a message.

[13:26 - 13:28] You want me to just send it in the chat to the interface? - Yes.

[13:29 - 13:37] - And just, 'cause there was an example that I looked at on Medium, maybe that's the way I can find it for Medium. That was the one on Medium yesterday.

[13:38 - 13:39] Oh yeah, here we go. I found it here.

[13:40 - 13:43] Oh, I'll put this in the chat here real quick. Okay.

[13:44 - 13:47] Yeah, so it's called layout LM. All right, so the name of the piece.

[13:48 - 13:57] Is that what they use in that other, the other code word? - So these one, I already disclosed these one on the document processing catalog.

[13:58 - 14:05] - Oh, it's catalog, the layout LM V3. They were using some kind of EPI from the Microsoft Azure. - Okay.

[14:06 - 14:12] - Got it. - Yeah. - So these, I don't think these will be good for your use case.

[14:13 - 14:19] - For my use case. - Okay, all right. - But what I was talking about was from the Azure EPI.

[14:20 - 14:33] - The Azure EPI. - Pretty good. - Yeah. And the downside of it is I asked him to show the invoice and the invoice was pretty, the cloud invoice, the cost of processing those documents was high, yeah.

[14:34 - 14:40] So that's why I recommended you dock ETL says that it's more on our side and not on that side. - Yeah.

[14:41 - 14:45] - Yeah. - No, that's fair. 'Cause I can tell all these tools, everybody wants a piece of it.

[14:46 - 14:54] Everything is used, whether it's blank views or cohere or any of them, you're all getting dinged every call. - Yes.

[14:55 - 15:03] - Also, you can try the redupto. If you don't wanna implement your own processing with redupto.ai. - Yeah.

[15:04 - 15:12] I looked at that one yesterday. I uploaded a test document and I sent one of those like ASICs shoes up there and I cut it down to only be like 13 pages.

[15:13 - 15:19] It was interesting though, in the playground, it only read one product page and then stopped. And I didn't know why.

[15:20 - 15:28] And I was doing the dock ETL process on the other side and I was like, let me just finish dock ETL. And then I got an email from reducto saying, hey, you signed up.

[15:29 - 15:36] Tell us what you're trying to do. So I may just reach out to them and say, hey, here's a sample of stuff I'm trying to parse.

[15:37 - 15:48] You guys tell me how best to do it, what it might cost since I see that. - The redupto.ai was used by one of the production applications, the text filing application.

[15:49 - 15:53] So that's why I suggested that. But I think what you're saying is correct.

[15:54 - 16:09] You might have to talk with them in order for them to fine tune that thing with your specificity. But if dock ETL is like working rather than like ET person, like on your three sort of, it's working better than those trust.

[16:10 - 16:13] Then you should just go with dock ETL. You shouldn't rely on the APIs.

[16:14 - 16:22] - Yeah, no, it makes sense. - Let me ask you one other question about dock ETL since you've used it a lot. Let me share my desktop real quick.

[16:23 - 16:30] Let me find a file that I think is pretty, pretty. So I wanted to ask you, so here's a great example of Hoka running shoes, right?

[16:31 - 16:37] The first part of this document, they talk about board technologies. - Now that could be useful for me, right?

[16:38 - 16:44] 'Cause every one of these shoes is gonna use some portion of this. It's an interesting feature to explain.

[16:45 - 17:02] What I really care about is when I get into, this is just an explanation about what each of these shoes, it's for racing or everyday or recovery, right? But then they start getting into the individual lines and this is the meat of what matters.

[17:03 - 17:17] So what I was wondering with dock ETL, maybe I have to do it in a couple of passes 'cause what I noticed when I ran it, it read every one of these pages. It classified them as table of contents or something, but then like you could see it didn't capture all the data.

[17:18 - 17:37] So do you know if dock ETL you can actually in one pass, give it like different schemas and say, hey, if you see this, do this, or you'd have to run it through like twice and say, yes, yes, you have to run it through different times in order for it to focus on different things. And then maybe you can aggregate the data using a LLM model.

[17:38 - 17:48] - Yeah, okay. - And this was the one, so I sent you on the chat. I think this is the one, this is the product that he used.

[17:49 - 17:57] - After the AI document and thought it's okay. All right, oh, look at that, oh, that looks good. - Yeah, it's amazing to me.

[17:58 - 18:03] Like there's so many different providers, libraries. It's almost never ending quite.

[18:04 - 18:08] - Yeah, that's right, at this point. - Yes. Can you show me the PDF one more time?

[18:09 - 18:17] The initial page. So I was wondering if it's gonna have the product name and the features.

[18:18 - 18:24] So it's not, it's gonna have just, so how you are gonna, my question is, - How would you match? - Yes.

[18:25 - 18:38] - No, I think you'd have to go down to the product page, right, where it says it's got something in here. Age frame, right, and it's probably one of those features that's in it, let's come back and look.

[18:39 - 18:45] Let's see if it's age frame is up here. Car and play matrix chord checks.

[18:46 - 19:04] Age frame, yeah, so it, it almost have to be like a, I don't, like what we were talking yesterday and that chat TPT recommendation about creating like assertions, like instead of looking at it all with one product, you have these individual like facts, right? Yes, which is, here's a fact at age frame and this.

[19:05 - 19:20] And so then I think if I'm in the creative process, right, I say, okay, write a good description. If you don't know what age frame, go look at the database, the brand technology database for hooker or something like that, oh.

[19:21 - 19:43] - So the, maybe we can use, so in this case, if you want to pass through in single pass and you do not, if you don't have the consideration, of course, for the time see, maybe you can experiment this directly with the LLM model, the chat GPT model. We will first try, like you don't need to try it like on the APIs or anything.

[19:44 - 20:06] What you can do is you can upload this on the chat GPT and then use prompt engineering technique in the chat itself in order for it to output the product, in order for it to output the schema, JSON schema in the chat itself. And if you are satisfied with those results, then we can implement the chat GPT in here.

[20:07 - 20:17] - So let me make sure I understood that. So you're saying like, just copy and paste some of this text out? - No, just attach the PDF in the chat GPT.

[20:18 - 20:26] - Oh, yes. - Give it the whole PDF. And then write me a product description and specifications about.

[20:27 - 20:30] - The schema. - The whole issue or something. - The schema with the JSON output.

[20:31 - 20:39] And then maybe we can have some Twitch drone on LDD speed high queries. And then see the results.

[20:40 - 20:50] If the results are good, then basically we can stop using the doc detail and directly use a single pass of charge GPT. - But again, so I make sure I understood.

[20:51 - 20:57] - You mean like literally attacks the PDF in a chat GPT conversation? - Yes, exactly.

[20:58 - 21:06] - And then tell what structured output for a product? - Yes. So something like this, let me stop sharing.

[21:07 - 21:13] - Oh yes, release the new technology, which is gonna slice the GPU consumption for a model. - Oh, really?

[21:14 - 21:24] - Nice. - So in here you can basically attach the PDF. And then you can just make sure the product is correct in here, and then schema, and then the JSON file.

[21:25 - 21:29] And see if it's outputting what we want. - Okay.

[21:30 - 21:36] - And now I can just attach as an attachment, or paste it in too. - Just the attachment. - Just the attachment, okay.

[21:37 - 21:46] - Yeah. Once we know, so make sure the testing is not for single product, it's for at least a product. Yes, or full products.

[21:47 - 22:10] And then you can also ask in the chat itself that if it does not fit in one response, try to give multiple response, try to fit it in multiple response at the end. And then once we see if it's performing better on the chat side, then for sure we will know that it will perform better on the API side, because we will have some things on the API.

[22:11 - 22:16] - 'Cause we could do more with the prompt and everything. - Yes, and then it's gonna be single, please. - Yeah.

[22:17 - 22:22] The only thing I was worried about is the files are big. There are 120-something pages, sometimes multiple megabytes.

[22:23 - 22:33] So I noticed like with some of those other tools, landing AI, I was showing you yesterday, like there would be like this files too big, it's over 50 pages or something like that. - Yes.

[22:34 - 22:36] - Yeah. - Yeah. - That might be...

[22:37 - 22:42] - But don't try it. That's a good suggestion. Honestly, I've just only used the chat interface.

[22:43 - 22:48] I haven't attached a bunch of documents to a conversation to see what it can do. - Yes.

[22:49 - 23:13] Maybe once we know if it's working good, then even the single, I don't think the single pass is a nice approach, because when you say like 200, 200 pages, I know for sure that these LLM models are designed in a way to skip some context. And if we just do a single pass, it's not gonna be a reliable approach in their long term.

[23:14 - 23:18] So maybe you can stick with the dark ETL. - And just have multiple passes, depending on what I'm looking to capture and everything.

[23:19 - 23:29] - Yes. - It has the full context of each passes. - Yeah, I think what I realized is maybe the hard part is the look up, okay?

[23:30 - 23:49] - Once you have this, it's the matching, it's like the index. I spent a lot of time yesterday, like with TurboPuffer, it'd be M25, tech searches versus vector searches. And I just think for the accuracy to be high, 'cause that's what I was telling you, like the clients worried about, they're worried about.

[23:50 - 23:57] For example, they get specifications wrong on a product, and then the customer wants to return it, 'cause they said he didn't match. And I have another customer who sells electrical equipment.

[23:58 - 24:04] So if you get a spec wrong on an electrical breaker or wire, you can have a building fire in it. It's really a big problem.

[24:05 - 24:06] - Yes. - It has to be good.