Lecture 02

2023-11-21
  • Section 01 (Cont.)
    • System cycles
      • Bandwidth vs. computation
      • High bandwidth => Delegate job to another machine
      • Cheap computation => thick client
    • Grid computing:
      • For improving throughput
      • Grid system architecture, standards and middleware
      • Overlay network (logical level) & IP network (Physical level)
        • Distribute system does not share memory
    • Cloud & IoT
      • IoT: A thing with processor and network hardware
      • Three layers: Cloud, Edge and Sensors (Controllers)
      • Fog: layer between cloud and edge
      • Numbers of devices: Edge => Billions, Fog => millions, cloud => thousands
    • Distributed parallel computing system classification
      • Multicomputer cluster: Big box of computers
      • Pear-to-Pear: Not for high performance computing (PC's are too far away), easy to scale, but hard to manage, hard to know state of the whole system.
      • Client-Server: Server often becomes a bottleneck (does not scale), but easy to manage. Easy to take a picture of and coordinate system state.
    • Share memory vs. distributed memory
    • Processes operate on data, exchange data with each others
    • Two ways of communication
      • Shared memory: One process place data on some place, another process reads it. Memory is centralized and not scalable. Variable "A" here and "A" there are the same.
      • Distributed memory: Variable "A" here and "A" there are not the same. Communicate through network.
      • Splitting processes and memories introduces new complexity dimension
        • Splitting processes: HPC
        • Splitting memories: Big Data
    • Parallel and distributed programming
      • MPI: Message passing interface (focus on processes)
        • We are at a state with a lot of data (data querying)
      • MapReduce: a programing paradigm (not a software)
    • Distributed System: Definitions
      • Distributed vs. Paralleled
      • Not necessarily depends on physical layer (split processes / split memories / computer network)
      • Definition: Network + only communicate with message passing
      • Network layer review
      • Computer network vs. Distributed System
        • Data Transmission | Computer network infrastructure | Distributed System
    • Distributed System challenges
      • Transparencies: make some concepts invisible to a user