Technical Workshop - Transformers, Attention Mechanisms, and Large Language Models in Knowledge Graph Embedding (1st Offering) by: Amangel Bhullar

 

Transformers, Attention Mechanisms, and Large Language Models in Knowledge Graph Embedding (1^st Offering)

Presenter: Amangel Bhullar

Date:  Wednesday, June 11^th, 2025

Time: 12:00 pm

Location: Workshop Space, 4th Floor - 300 Ouellette Ave., School of Computer Science, Advanced Computing Hub

 

Knowledge graphs (KGs) have become foundational in representing complex, structured relationships among entities in diverse domains, from social networks to biomedical research. Traditional knowledge graph embedding methods rely on algebraic or geometric models. Still, recent breakthroughs leverage neural architectures, particularly transformers, attention mechanisms, and large language models (LLMs), to learn richer, context-aware embeddings. This workshop will introduce participants to the core concepts and recent advances in applying transformers and LLMs to knowledge graph embedding. We will explore how these techniques utilize both graph structure and textual semantics and examine real-world applications.

 

1. Introduction to Knowledge Graphs and Embedding
   • What are knowledge graphs?
   • Challenges in knowledge graph completion
   • Overview of embedding methods
2. Foundations of Transformers and Attention Mechanisms
   • Basics of the transformer architecture
   • Self-attention and its role in modeling graphs
   • Graph Attention Networks (GATs) and variants
3. Large Language Models for Knowledge Graphs
   • Introduction to LLMs (BERT, GPT, etc.)
   • Leveraging entity descriptions and textual context
   • KG-augmented transformers (K-BERT, KEPLER, etc.)
4. Integration: Combining Graph Structure and Language Models
   • Hybrid architectures
   • Training objectives and loss functions
   • Example pipelines
5. Applications and Case Studies
   • Link prediction and knowledge graph completion
   • Real-world use cases in social networks and beyond
   • Benchmark datasets
   • Evaluation metrics
6.  Discussion and Future Directions

• Research challenges and open problems
• Q&A

 

• Basic knowledge of machine learning and deep learning concepts.
• Introductory understanding of graphs and graph theory.

 

 

Amangel Bhullar is a Ph.D. candidate in Computer Science at the University of Windsor, specializing in artificial intelligence with a focus on knowledge representation, machine learning, Social networks, and knowledge graphs. She currently serves as the President of the Graduate Student Society (GSS), where she leads initiatives to enhance the academic and social experience of graduate students.

In addition to her role at GSS, Amangel is the Director of the Lancer Sport and Recreation Center (LSRC) Corporation and serves as a Member of the Board of Governors at the University of Windsor. Her leadership contributions extend further as an Ex-Officio Member of the University Senate, where she brings a student-centred perspective to university policy and governance matters.

Registration Link (only MAC students need to pre-register)