I previously tried (and failed) to setup LLM tracing for hinbox using Arize Phoenix and litellm. Since this is sort of a priority for being able to follow along with the Hamel / Shreya evals course with my practical application, I’ll take another stab using a tool with which I’m familiar: Braintrust. Let’s start simple and then if it works the way we want we can set things up for hinbox as well. Simple Braintrust tracing with litellm callbacks Callbacks are listed in the litellm docs as the way to do tracing with Braintrust. So we can do something like this: import litellm litellm.callbacks = ["braintrust"] completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], metadata={ # "project_id": "1235-a70e-4571-abcd-234235", "project_name": "hinbox", }, ) print(completion_response.choices[0].message.content) You can pass in a project_id or a project_name and the traces will be routed there. Here’s what it looks like in the Braintrust dashboard: Our first trace logged in Braintrust Note how you can’t see which model was used for the LLM call, nor any cost estimates. The docs mention that you can pass metadata into Braintrust using the metadata property: “braintrust_* - any metadata field starting with braintrust_ will be passed as metadata to the logging request” (link) This seems a bit rudimentary, however. If we take a look at the full tracing documentation on the Braintrust docs we can see that they seem to recommend wrapping the OpenAI client object instead: import os from braintrust import init_logger, traced, wrap_openai from openai import OpenAI logger = init_logger(project="hinbox") client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"])) # @traced automatically logs the input (args) and output (return value) # of this function to a span. To ensure the span is named `answer_question`, # you should name the function `answer_question`. @traced def answer_question(body: str) -> str: prompt = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": body}, ] result = client.chat.completions.create( model="gpt-3.5-turbo", messages=prompt, ) return result.choices[0].message.content def main(): input_text = "What's the capital of China? Just give me the name." result = answer_question(input_text) print(result) if __name__ == "__main__": main() This indeed does label the span as answer_question but it doesn’t do much else. Even the model name isn’t captured here. Instrumenting a series of calls to handle ‘deeply nested code’ (as their docs puts it) even didn’t log the things it was supposed to: import os import random from braintrust import current_span, init_logger, start_span, traced, wrap_openai from openai import OpenAI logger = init_logger(project="hinbox") client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"])) @traced def run_llm(input): model = "gpt-4o" if random.random() > 0.5 else "gpt-4o-mini" result = client.chat.completions.create( model=model, messages=[{"role": "user", "content": input}] ) current_span().log(metadata={"randomModel": model}) return result.choices[0].message.content @traced def some_logic(input): return run_llm("You are a magical wizard. Answer the following question: " + input) def simple_handler(input_text: str): with start_span() as span: output = some_logic(input_text) span.log(input=input_text, output=output, metadata=dict(user_id="test_user")) print(output) if __name__ == "__main__": question = "What's the capital of China? Just give me the name." simple_handler(question) This is adapted from the example they pasted in their docs as their one isn’t even a functional code example on its own. It is seeming increasingly clear that Braintrust isn’t going to be the right choice, at least as long as I want to keep using litellm. I know that Langfuse has a very nice integration with litellm, so I think I’ll pivot over to that now. Basic tracing with Langfuse and litellm Simple tracing is easy: import litellm litellm.callbacks = ["langfuse"] def query_llm(prompt: str): completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], ) return completion_response.choices[0].message.content def my_llm_application(): query1 = query_llm("What's the capital of China? Just give me the name.") query2 = query_llm("What's the capital of Japan? Just give me the name.") return (query1, query2) print(my_llm_application()) We specify langfuse for the callback and each llm call is logged as a separate trace + span. Here you can see what this looks like in the dashboard: Basic trace and span in Langfuse dashboard The litellm docs include information on how to specify custom metadata and grouping instructions for Langfuse. Notably, we can specify (as of June 2025, at least!) things like a session_id, tags, a trace_name and/or trace_id as well as custom trace metadata and so on. So we can get most of what we want to specify in the following way: import litellm litellm.callbacks = ["langfuse"] def query_llm(prompt: str, trace_id: str): completion_response = litellm.completion( model="openrouter/google/gemma-3n-e4b-it:free", messages=[ { "content": "What's the capital of China? Just give me the name.", "role": "user", } ], metadata={ "trace_id": trace_id, "trace_name": "my_llm_application", "project": "hinbox", }, ) return completion_response.choices[0].message.content def my_llm_application(): query1 = query_llm( "What's the capital of China? Just give me the name.", "my_llm_application_run_789", ) query2 = query_llm( "What's the capital of Japan? Just give me the name.", "my_llm_application_run_789", ) return (query1, query2) if __name__ == "__main__": print(my_llm_application()) This looks like this in the Langfuse dashboard: Spans grouped into traces This is honestly most of what I’m looking for in terms of my tracing. If I were to use a non-OpenRouter model, moreover, I’d also get full costs in the Langfuse dashboard, e.g.: LLM costs in Langfuse dashboard As such, I can monitor costs from within OpenRouter and have the option to keep track of costs in Langfuse by passing custom metadata should I wish. I’ll make a separate blog where I actually go into how I set up + instrumented hinbox for this kind of tracing while continuing to use litellm.

Google released an updated iteration of their Deep Research tool that uses the new 2.5 Pro model. This was taken from a post originally made on Twitter, so please excuse the terseness. First impressions: a bit too eager to jump into a deep research task even when I just ask a clarifying question quite verbose, just like the OpenAI version. Not sure why both play this up a lot. It looks impressive but in practice I think we need more entry points into this. The ‘Executive Summary’ and other concluding headers are nice touches but I feel maybe there should be some more adherence to user requests for short reports. (I get that as UI it’s maybe weird to think for 10 mins and then spit out a very concise version, but it might actually be more useful.) I continue to be annoyed about how these Gemini DR reports handle footnotes (i.e. as endnotes whacked on at the end of the report). Almost a deal-breaker IMO. It’s almost like GDR tries to show how scholarly and serious it is by giving you these walls of prose (vs OpenAI DR which throws in a lot more bullet points). Not sure one is better than the other but would appreciate a bit more flexibility! The portability of these reports has always been not great. Yes you can export them to Google Docs but markdown (+ other options) would have been much better. In practice, this means that whenever I use GDR the report stays stuck there and I’m far less likely to share it with anyone, whereas the OpenAI DR reports I drop parts/all into a Github Gist etc. These reports have been getting better and better, all things considered. I’ve been following along and using GDR from the early days (even pre-OpenAI DR) and this latest version is the best version of it so far (as you’d hope!) (It’s also a little bit annoying that GDR has removed any way to use the older versions of GDR with Gemini Pro 2.0 and 1.5 etc. Makes it harder to actually compare these things.) Please let’s get an ipad version of the Gemini iPad app soon, too? Feels a bit regressive to have to use GDR on the web interface always. For serious research (as opposed to simply generating a nice report on some area where you don’t know much about already), all these tools remain hamstrung by the quality of the sources. In areas where I am (or very recently used to be) a leading scholar / researcher, the difference between what I’d expect (in terms of taste / discernment for picking out these sources) is especially egregious. Make the models better, yes, but have better filters + retrieval. So yeah, these tools are getting good! Kudos to the teams who are implementing this stuff. Hard to make it perform reproducibly well on so many open-ended uses. But more work to be done! IMO the really great implementations of this ‘deep research’ pattern will all be in-house where you can have control over: source selection (i.e. high-quality inputs only, not just some random things on the internet) how long it spends thinking about a particular area / loop of the research (or decides to backtrack and dig deeper etc) output types / templates / length different modalities of Q&A (sometimes you want reports, other times you want a quick question answered, other times you want visual guides etc etc.) different models for different kinds of tasks possibly you have little sub-research agents / processes which will go off and work on some hypothesis, possibly involving actual datasets / analysis of tabular data etc, something clearly missing from the current versions we have A few other things: GDR’s ‘clarification step’ (which I’ve heard them discuss on podcasts etc) is not as good or useful as the OpenAI DR clarification questions. In practice, because it’s buried under a concealment button that you have to click etc, and where the entire UI seems to be screaming at you to ‘Start Research’, you basically never update or amend the research plan. And when you do, it’s really not clear what’s changed because you don’t get some feedback or diff that your comments were understood; you just get an entire new research plan (again buried under the concealment button) Going forward we’re probably going to want / need ways of navigating the layers to this research. A global overview report will have subsections that (should you wish) can be expanded into their own more detailed or granular reports. This is how research works, after all. Not just endless new reports all trailed one after another pointed in the same direction. The other thing that I think we’re really going to need to work on is research taste. Like the LLMs that power them, GDR and OpenAI DR offer a level of research taste developed to the mean. (I know people are thinking about this since it came up on Dwarkesh’s podcast with the AI 2027 guys, but they were focused on scientific research.) I think there’s not a single answer for this which is, again, why I see the end result as people bringing these things in-house where they get to develop and refine what makes their particular flavour of research unique. (In the human-generated research world this is very much the case, where certain institutions (or even particular authors) are known for how deep they go, or what kinds of sources they prefer, or how they choose to feature or highlight the primary sources they access, and so on.) There are many possible variations of how this manifest, and I hope that we’re headed into a world where all the AI ‘deep researchers’ will be unique and quirky in all the best senses of that word.

I took the past week off to work on a little side project. More on that at some point, but at its heart it’s an extension of what I worked on with my translation package tinbox. (The new project uses translated sources to bootstrap a knowledge database.) Building in an environment which has less pressure / deadlines gives you space to experiment, so I both tried out a bunch of new tools and also experimented with different ways of using my tried-and-tested development tools/processes. Along the way, there were a bunch of small insights which occurred to me so I thought I’d write them down. As usual with this blog, I’m mainly writing for my future self but I think there might be parts that are useful for others! Apologies for the somewhat rushed nature of these observations; better I get the blog finished and published than not at all! 🤖 Local Models During this project, I experimented with several local models, which continue to impress me with their evolving capabilities. The recent launch of gemma3 was particularly timely - I found myself regularly using the 27B version, which performed admirably across various tasks. There are three or four models I keep returning to. mistral-small stands out as an exceptional model that’s been relatively recently updated and seems a bit underrated / underappreciated. The original mistral model continues to hold up remarkably well, particularly for structured extraction tasks and general writing needs like summarization. One important realization when working with real-world use cases: benchmarks can be deceptive. While helpful as general indicators, each model has its own strengths and quirks. Many newer models are heavily optimized for structured data extraction, but their performance ultimately depends on whether their training documents align with your specific use case. It’s crucial to test models against your actual requirements rather than relying solely on published benchmarks. For robust results with local models, I’ve found that implementing a “reflection, iterate and improve” pattern significantly enhances performance. When you need a model to summarize or analyze content in a particular format, having a secondary model (or even the same model!) review the output against the original prompt requirements is incredibly valuable. This reviewer model can suggest improvements to better fulfill the original request. Running this loop for 2-5 iterations (depending on complexity) can yield results approaching those of proprietary models like Claude or GPT-4, which might achieve similar quality in a single pass. For local deployments, this iterative improvement pattern is essentially non-negotiable. I also explored vision models, particularly llava and llama-3.2-vision. These were my primary tools for extracting context from images, generating captions, and analyzing visual content. Their effectiveness varies based on content type and language, but they represent impressive capabilities that can run entirely on local systems. A significant portion of my work involved non-English languages, including some relatively rare ones. This is another area where benchmark claims about supporting “hundreds of languages” often don’t align with real-world performance. Models might list impressive language coverage in their specifications, but actual proficiency varies dramatically. It reinforces my earlier point - always verify benchmark claims against your specific use case before committing to a particular model. 💬 Prompting & Instruction Following Working extensively with various models during this project reinforced some fundamental insights about prompting that might seem basic, but prove critical in practical applications. These observations are particularly relevant when working with local models, though they apply to cloud-based systems as well. Context matters significantly more than we might assume. While we’ve grown accustomed to proprietary models like Claude or GPT-4o performing admirably with minimal guidance, local models require more deliberate direction. The more relevant context you can provide (within reasonable token limits), the better your results will be. If you would naturally provide certain background information to a human performing the task, make sure to include it in your prompt to the model as well. Another key insight: every model has its unique characteristics. Techniques that work brilliantly with one model might fall flat with another, especially in the local model ecosystem. They each require slightly different prompting approaches, specific phrasing patterns, and tailored guidance. This necessitates running small experiments to understand how different models respond to various prompting styles. It’s still more art than science, but this experimentation phase is crucial when implementing local models effectively. Perhaps the most valuable lesson I rediscovered is that breaking complex tasks into smaller components yields superior results compared to using a single comprehensive prompt. This is particularly true with local models. When performing extensive data extraction or when dealing with structured data where the extraction targets differ significantly from each other, don’t expect the model to handle everything in one pass – even a human might struggle with such an approach. Instead, break down the task into logical components, create targeted mini-prompts for each aspect, and then recombine the results once all the separate LLM calls are completed. Yes, this approach adds processing time and complexity, but the quality improvement is well worth the trade-off. When accuracy matters more than speed, this decomposition strategy consistently delivers better outcomes. 🧰 Process & Tools My development environment during this project provided plenty of opportunities to evaluate various tools and workflows. As context, I primarily work on a Mac while maintaining access to a separate (local) machine with GPU capabilities for more intensive tasks. This setup allows me to flexibly experiment with both local and cloud-based models. For managing local models, Ollama continues to be my go-to solution for downloading, running, and interfacing with these models. A recent discovery that significantly improved my workflow is Bolt AI, an excellent Mac interface that provides seamless switching between local Ollama models and cloud-based alternatives. If you’re working in a hybrid model environment, Bolt AI is definitely worth exploring. I’ve also recently integrated OpenRouter into my toolkit, which solves the problem of managing countless API keys across different inference providers. OpenRouter not only offers native connections to many cloud providers but also allows you to incorporate your own API keys, streamlining access to a diverse model ecosystem through a unified interface. It also helps with setting spend limits on various models or projects. In terms of development insights, I was impressed by how rapidly front-end development can progress with the assistance of models like Claude 3.7 and OpenAI’s O1-Pro. These models perform exceptionally well when supplemented with documentation (such as an llms.txt file) alongside your prompts. While I can’t speak to their effectiveness with extremely complex applications or massive frontend codebases, they demonstrate remarkable proficiency with small to medium-sized projects. A significant portion of my experimentation involved RepoPrompt, a tool that recently transitioned from free beta to a paid license model. RepoPrompt addresses the challenge of getting your codebase into an LLM-friendly format. Unlike standard CLI tools that simply export code to clipboard or text files, RepoPrompt generates a structured XML representation that, when modified by an LLM and pasted back, creates a reviewable diff of the proposed changes. At least, that’s one of the things it allows you to do! It’s actually a bit more powerful / flexible than that and here’s a video so you can see it in action: RepoPrompt Demo Video While tools like Cursor and Windsurf offer similar functionality, they tend to become less reliable as project complexity increases. RepoPrompt shines when paired with an OpenAI Pro subscription, enabling effective integration of models like O1 Pro and o3-mini-high into your development lifecycle. In my testing, the RepoPrompt + O1 Pro/O3 Mini High combination consistently delivered superior results compared to using Cursor with Claude 3.7 (even with ‘Thinking Mode’ enabled). Despite the occasional pauses while these models process complex problems, the quality improvement justifies the wait. Additionally, I continued working with Claude Code and CodeBuff, both CLI-driven tools focused on code improvement. Of the two, CodeBuff has become my preferred option. Both tools require careful supervision—I typically keep Cursor open to monitor changes in real-time, occasionally needing to revert modifications or redirect the approach. These tools excel when you clearly articulate your objectives and maintain oversight of the implementation process. CodeBuff particularly impresses with larger codebases and demonstrates superior stability overall. An interesting pattern emerged during development: whenever files approached 800-900 lines, it signaled the need to refactor into smaller submodules to maintain LLM comprehension, especially when using agent mode in Cursor. The modular approach significantly improved model performance. I was genuinely surprised by the effectiveness of the RepoPrompt and O1 Pro combination. For smaller, targeted modifications, CodeBuff continues to demonstrate remarkable capability. While I didn’t evaluate these tools in conjunction with local models, I suspect such combinations would require more iterative refinement to achieve comparable results. 🧑‍🔬 Software Engineering Patterns Throughout this experimental project, several software engineering principles proved particularly valuable when working with LLM-assisted development. These patterns aren’t revolutionary, but their importance amplifies in the context of AI-augmented workflows. The principle of simplicity served as a cornerstone approach. Breaking development into the smallest logical next task repeatedly demonstrated its value, especially during the exploratory phases when project architecture was still taking shape. While some engineers might possess the cognitive bandwidth to fully conceptualize complex systems with perfect abstractions from the outset, I’ve found incremental development leads to more robust outcomes. This approach aligns naturally with how most developers actually think through problems and provides clear checkpoints for evaluating progress. Data visibility emerged as another critical factor. When leveraging LLM-assisted coding, comprehensive logging becomes even more essential than in traditional development. Strategically placed log outputs create a diagnostic trail that proves invaluable when troubleshooting unexpected behaviors. This practice creates a feedback loop that strengthens both your understanding of the system and the LLM’s ability to assist effectively. A particularly underappreciated practice I haven’t seen widely discussed is the importance of dead code detection. When working with LLM-assisted development, code cruft tends to accumulate more rapidly than in conventional programming. Tools like deadcode and vulture provide static analysis of Python projects to identify unused functions and variables. Running these tools periodically helps maintain codebase clarity by flagging remnants that might otherwise cause confusion during review. I’m not certain whether newer tools like ruff from Astral include this functionality (particularly for function calls), but the capability is invaluable for maintaining a clean, navigable codebase. Taking time to think offline—away from the keyboard—often yields surprising clarity. This deliberate pause creates space to articulate precisely what you need for the next development increment. When you can express your requirements with precision, the LLM’s output improves proportionally. Ambiguous instructions inevitably produce suboptimal results, whereas clarity fosters efficiency. A final observation worth emphasizing: having experience as an engineer in the pre-LLM era remains tremendously advantageous. When confronting complex workflows involving chained LLM calls with interdependencies and reflection patterns, traditional debugging skills become indispensable. Knowing when to step away from AI assistance and dive into manual debugging with tools like pdb, stepping through code execution and inspecting variables directly, represents a crucial judgment call. LLMs and coding agents often demonstrate a bias toward generating new code rather than methodically analyzing existing problems. Recognizing the moment when direct human intervention becomes more efficient than continually prompting an AI is a skill that comes with experience. Once you’ve manually identified the underlying issue, you can return to the LLM with precisely targeted prompts that yield superior results. 🌐 Appendix 1: FastHTML As a practical addition to my experimentation, I implemented FastHTML for the first time to build a frontend for my knowledge base extraction assistant. The experience was remarkably frictionless, particularly when leveraging their llms.txt file—a markdown-formatted documentation set that integrates seamlessly with your frontend codebase when provided alongside prompts. This approach works exceptionally well with models like O1 Pro or O3 Mini High, creating a development workflow that feels intuitive and responsive. Despite having substantial JavaScript experience from previous roles, I found FastHTML significantly more manageable than complex JavaScript frameworks that dominate the ecosystem today. The reduced cognitive overhead and natural integration with Python-based workflows makes FastHTML a compelling choice for ML practitioners who prefer to minimize context-switching between languages and paradigms. The framework strikes an excellent balance between capability and simplicity that aligns perfectly with rapid prototyping and iterative development cycles common in ML projects. For those building interfaces to ML systems, it’s definitely worth considering as your frontend solution. 📃 Appendix 2: OCR + Translation Another interesting challenge I tackled involved OCR and translation of handwritten documents in non-English languages—a task that proved impossible to accomplish in a single pass with local models, particularly for less common languages. The solution emerged through methodical problem decomposition: Breaking down PDFs into individual page images Segmenting each page into overlapping image chunks (critical for handwriting where text may slant across traditional line boundaries) Applying OCR to extract text in the original source language from each image segment Using translation models to convert the extracted text to English This multi-stage pipeline allowed me to overcome the limitations of local models when confronted with the combined complexity of handwriting recognition and translation. Both gemma3 and llama-3.3 performed admirably within this decomposed workflow, demonstrating that even resource-constrained local deployments can achieve impressive results when problems are thoughtfully restructured. This case exemplifies a core principle of effective ML implementation: when dealing with complex, multi-faceted challenges, breaking them into targeted sub-problems often yields better outcomes than attempting end-to-end solutions—especially when working with constrained computational resources. While this approach may increase processing time, the quality improvement justifies the trade-off for many practical applications.

Finally back on track and reading the next chapter of Chip Huyen’s book, ‘AI Engineering’. Here are my notes on the chapter. Overview and Core Philosophy “Data will be mostly just toil, tears and sweat.” This is how we start the chapter :) This candid assessment frames dataset engineering as a discipline that requires both technical sophistication and pragmatic persistence. While the chapter’s placement might have been suitable earlier in the book, its position allows it to build effectively on previously established concepts. Data Curation: The Foundation Data curation addresses various use cases including fine-tuning, pre-training, and training from scratch, with specific considerations for chain of thought reasoning and tool use. The process addresses three fundamental aspects: Data Quality: The equivalent of ingredient quality in cooking Data Coverage: Analogous to having the right mix of ingredients Data Quantity: Determining the optimal volume of ingredients Quality Criteria Data quality encompasses multiple dimensions: Relevance to task requirements Consistency in format and structure Sufficient uniqueness Regulatory compliance (especially critical in regulated industries) Coverage Considerations Coverage involves strategic decisions about data proportions: Large language models often utilize significant code data (up to 50%) in training, which appears to enhance logical reasoning capabilities beyond just coding Language distribution can be surprisingly efficient (even 1% representation of a language can enable meaningful capabilities) Training proportions may vary across different stages of the training process Quantity and Optimization A key phenomenon discussed is ossification, where extensive pre-training can effectively freeze model weights, potentially hampering fine-tuning adaptability. This effect is particularly pronounced in smaller models. Key quantity considerations include: Task complexity correlation with data requirements Base model performance implications Model size considerations (OpenAI notes that with ~100 examples, more advanced models show superior fine-tuning performance) Potential for using lower quality or less relevant data for initial fine-tuning to reduce high-quality data requirements Recognition of performance plateaus where additional data yields diminishing returns Data Acquisition Process The chapter provides a detailed example workflow for creating an instruction-response dataset: Initial dataset identification (~10,000 examples) Low-quality instruction removal (reducing to ~9,000) Low-quality response filtering (removing 3,000) Manual response writing for remaining high-quality instructions Topic gap identification and template creation (100 templates) AI synthesis of 2,000 new instructions Manual annotation of synthetic instructions Final result: 11,000 high-quality examples Data Augmentation and Synthesis Synthesis Objectives Increasing data quantity Expanding coverage Enhancing quality Addressing privacy concerns Enabling model distillation Notable Research: An Anthropic paper (2022) found that language model-generated datasets can match or exceed human-written ones in quality for certain tasks. Note that some teams actually prefer AI-generated preference data due to human fatigue and inconsistency factors. Synthesis Applications The chapter distinguishes between pre-training and post-training synthesis: Synthetic data appears more frequently in post-training Pre-training limitation: AI can reshape existing knowledge but struggles to synthesize new knowledge LLaMA 3 Synthesis Pipeline A comprehensive workflow example: AI generation of problem descriptions Solution generation in multiple programming languages Unit test generation Error correction Cross-language translation with test verification Conversation and documentation generation with back-translation verification This pipeline generated 2.7 million synthetic coding examples for LLaMA 3.1’s supervised fine-tuning. Model Collapse Considerations The chapter addresses the risk of model collapse in synthetic data usage: Potential loss of training signal through repeated synthetic data use Current research suggests proper implementation can avoid collapse Importance of quality control in synthetic data generation Model Distillation Notable example: BuzzFeed’s fine-tuning of Flan T5 using LoRa and OpenAI’s text-davinci-003 generated examples, achieving 80% inference cost reduction. Data Processing Best Practices Expert Tip: “Manual inspection of data has probably the highest value to prestige ratio of any activity in machine learning.” - Greg Brockman, OpenAI co-founder Processing Guidelines The chapter emphasizes efficiency optimization: Order optimization (e.g., deduplication before cleaning if computationally advantageous) Trial run validation before full dataset processing Data preservation (avoid in-place modifications) Original data retention for: Alternative processing needs Team requirements Error recovery Technical Processing Approaches Deduplication strategies include: Pairwise comparison Hashing methods Dimensionality reduction techniques Multiple libraries are referenced (page 400) for implementation. Data Cleaning and Formatting HTML tag removal for signal enhancement Careful prompt template formatting, crucial for: Fine-tuning operations Instruction tuning Model performance optimization Data Inspection The chapter emphasizes the importance of manual data inspection: Utilize various data exploration tools Dedicate time to direct data examination (recommended: 15 minutes of direct observation) Consider this step non-optional in the process