IVQ 1-50
Q1. How do attention mechanisms work in transformers, and why are they central to LLM performance?
Q2. What is the role of positional encoding in transformer-based models like LLMs?
Q3. How do LLMs represent and generate human language using embeddings?
Q4. What is a context window in LLMs, and how does it affect model performance and memory?
Q5. What is the difference between pre-training and fine-tuning in the context of large language models?
Q6. What are query, key, and value vectors in the attention mechanism, and how do they interact?
Q7. How does multi-head attention improve the model's ability to understand context compared to single-head attention?
Q8. Why is self-attention important in processing sequences, and how does it differ from traditional RNN approaches?
Q9. How are attention scores computed, and what role does the softmax function play in this process?
Q10. What are the computational challenges of attention in long sequences, and how do architectures like Longformer or FlashAttention address them?
Q11. How does increasing the context window size impact an LLM’s performance and computational requirements?
Q12. What are the limitations of a fixed context window, and how do models like Claude or Gemini handle extended context?
Q13. In what ways does context fragmentation affect coherence in multi-turn conversations with LLMs?
Q14. How do retrieval-augmented generation (RAG) methods help overcome context window limitations?
Q15. What strategies are used to compress or summarize prior context to fit within an LLM’s token limit?
Q16. How does parameter-efficient fine-tuning (PEFT) differ from full fine-tuning in terms of resource usage and model performance?
Q17. What role do low-rank matrices play in LoRA, and why are they effective for LLM adaptation?
Q18. How does QLoRA enable fine-tuning large models on consumer-grade hardware?
Q19. What are the trade-offs between using quantization-aware fine-tuning methods like QLoRA versus traditional approaches?
Q20. In which scenarios would you choose LoRA, QLoRA, or full fine-tuning for adapting a foundation model?
Q21. What are the trade-offs between beam search and nucleus sampling in text generation?
Q22. How does top-k sampling introduce diversity in LLM outputs, and when is it preferred over deterministic decoding?
Q23. What are the limitations of greedy decoding in capturing coherent long-form text?
Q24. How do temperature settings influence randomness and creativity in sampling-based decoding methods?
Q25. In what scenarios would you prioritize decoding efficiency over generation diversity in LLM applications?
Q26. How does adjusting the temperature parameter affect the randomness and determinism of LLM-generated text?
Q27. In what situations would a low-temperature setting be more appropriate than a high-temperature one?
Q28. How does temperature interact with top-k and top-p sampling in shaping output diversity?
Q29. What are the risks of using a high temperature in safety-critical LLM applications?
Q30. How can tuning temperature help balance creativity and coherence in generative tasks like storytelling or summarization?
Q31. How does causal language modeling differ from masked language modeling in LLM training?
Q32. Why is next sentence prediction used in some pretraining objectives, and what are its limitations?
Q33. How does span masking improve over token-level masking in models like SpanBERT?
Q34. What are the benefits and trade-offs of using denoising autoencoding for pretraining language models?
Q35. How does the choice of pretraining objective impact downstream performance on tasks like QA or summarization?
Q36. How do encoder-decoder architectures work in sequence-to-sequence models?
Q37. What makes sequence-to-sequence models suitable for tasks like machine translation and summarization?
Q38. How does attention enhance the performance of traditional sequence-to-sequence models?
Q39. In what ways do transformer-based seq2seq models outperform RNN-based ones?
Q40. How are sequence-to-sequence models used in speech recognition and text-to-speech systems?
Q41. What are the key differences between encoder-only, decoder-only, and encoder-decoder transformer architectures?
Q42. How does causal attention enable autoregressive language modeling?
Q43. Why are masked models like BERT better suited for understanding tasks than generation tasks?
Q44. How do bidirectional context models differ from left-to-right models in their ability to capture dependencies?
Q45. In what scenarios would you choose an autoregressive model over a masked language model, and vice versa?
Q46. How do word embeddings differ from contextual embeddings in modern language models?
Q47. What role do positional embeddings play in transformer architectures?
Q48. How are learned embeddings updated during pretraining and fine-tuning?
Q49. What is the significance of embedding dimensionality in LLM performance and memory usage?
Q50. How can pre-trained embeddings be reused or adapted in downstream NLP tasks?
Last updated