Lecture 01

2022-03-15
  • Textbooks:
    • Distributed Systems by George Coulouris (Or by Tanenbaum)
    • Master Cloud Computing by Buyya
  • Lecture + Lab
    • Lecture: fundamentals & theory
    • Lab: specific technology (not necessary linked to lecture)
      • MPI, Big Data System, etc.
  • Current state of cloud computing (Big machine)
    • Trends in computing
      • All about scalability
      • Scale-down: make system small, maximize device capacity
      • Scale-in: minimize end-to-end latency
      • Physical limits
    • Current state:
      • Petascale computing 10^15 float operation per second (FLOPs)
      • Multicore computing: 1-400 cores
    • Future goal:
      • Exascale computing 10^18 FLOPs
      • Manycore computing: 1000 cores
    • History: Mainframe (1950), Clusters (1970), Grids (2000), Clouds (2008)
      • 1966: SISD, SIMD, MISD, MISD (I - Instruction, D - Data)
        • SISD: Single instruction on single piece of data
        • SIMD: Same instruction on multiple pieces of data
        • MISD: not much sense in practice
        • MIMD: Multiple instructions (in parallel) on different pieces of data
          • What is cloud: An MIMD
      • First super computer: Manchester ATLAS 1962
    • Exascale computing among countries
      • China vs. USA and Euroupe
      • Top 500 performance list
      • Super computer in Fugaku is based on ARM
      • Sunway is based on processors designed and made in China
    • Drivers:
      • Traditional Workloads: Math
      • Data: size of data, coming speed of data
    • Moore's Law (is dead)
      • Cooling problem: Microsoft server under water
      • Energy is needed when transferring data (not processing)
      • Attempt1: Move from Compute-centric model to Data-centric model
      • Attempt2: Design new processor architecture
      • Attempt3: Quantum computing
  • System cycles of centralization and decentralization
    • (Centralization) no PC, only time-sharing machine
    • (Decentralization) PC and client-server design
    • (Centralization) Utility computing: Grid, Cloud and thin clients
    • (Decentralization) IoT and edge
    • (Centralization?) 5G
  • Grid (For scientific computing), Cloud (A bunch of devices), Fog (Cloud comes down, small devices become part of cloud)
  • Purpose of distributed system
    • High performance computing: Execute one job very fast
    • High throughput computing: execute many small jobs at the same time