Objectives and competences
The objective of this course is for students to be able to demonstrate understanding of information theory and communication model, to analyze the components of communication model, to use them in practice, and to design new variants of coding algorithms.
Content (Syllabus outline)
• Introduction: definition of information, code, coding, and communication system.
• Entropy: entropy of a discrete random process, the entropy of a discrete variable, Fano’s inequality, the entropy of a continuous variable.
• Information: average information, mutual information of two discrete variables, mutual information of two continuous variables.
• Discrete information sources: entropy of stationary source, ergodic stationary source, memoryless source, a source with memory, redundancy.
• Communication channels I: discrete communication channels, discrete communication channel capacity, continuous communication channels, continuous communication channel capacity.
• Information source coding: fixed-length and variable-length coding, Kraft inequality, Huffman codes, LZW code, arithmetic codes, RLE code.
• Secrecy coding: cryptosystems with secret key, DES and AES cryptosystems, cryptosystems with public key, RSA cryptosystem, digital signature.
• Communication channels II: error-detecting codes, error-correcting codes, optimal decoding, Shannon theorem of secure coding, inversion of Shannon theorem.
• Secure coding: linear block codes, Hamming codes, cyclic codes, Golay codes, Reed-Muller codes, convolutional codes.
Learning and teaching methods
• Lectures: in lectures, students get to know the theoretical contents of the course. Lectures are conducted as classical lectures in frontal form, interleaved with discussions.
• Tutorials: in tutorial exercises, students reinforce theoretical knowledge by solving practical tasks in collaboration.
• Lab work: in laboratory exercises, students apply acquired knowledge on problems from telecommunications. They improve their transferable skills by using an integrated development tool.
• Homework: in tutorial exercises, students reinforce theoretical knowledge by solving practical tasks independently.
Intended learning outcomes - knowledge and understanding
On completion of this course the student will be able to
• explain the theoretical basis of building the mathematical model of communication system phenomena,
• explain the theoretical basis of information and other parameters for description of communication and information systems,
• select the best coding algorthm and cryptographic method for requested problem,
• design new variants of coding algorithms.
Intended learning outcomes - transferable/key skills and other attributes
• Communication skills: oral defence of computer skills, manner of expression at the written exam.
• Use of information technology: use of software tools and programming environments.
• Problem-solving: problem analysing, designing algorithms, coding, and testing programs.
• Working in a group: organisation and leading of a group, active collaboration
Readings
• N. Pavešić: Informacija in kodi, Univerza v Ljubljani, druga izdaja, Založba FE in FRI, Ljubljana, 2010.
• T. M. Cover, J. A. Thomas: Elements of Information Theory, Wiley-Interscience, New York, 1991.
• R. Wells: Applied Coding and Information Theory for Engineers, Prentice-Hall, Upper Saddle River, 1999.
• D. Welsh: Codes and Cryptography, Oxford University Press, New York, 1989.
Prerequisits
Recommended is the basic knowledge of mathematics and probability theory.Conditions for course completion: to attend the written exam, the student must have at least 80% attendance on the laboratory work and a positive grade from the laboratory work.
Additional information on implementation and assessment The written exam may be substituted with continuous knowledge testing and assessment.
