System Administration & IT Infrastructure Services

The Role of a Sys Admin

• System administration is the field in IT responsible for maintaining reliable, multi-user computing environments — the hardware, networks, and services that a company depends on.
• IT infrastructure encompasses hardware, networking, and services such as email, file storage, and web hosting. These typically run on servers.
• Machines that consume server resources are called clients. The sys admin manages the servers so clients can reliably reach them.
• Common server form factors:
  • Tower servers — look like desktop PCs; standalone units
  • Rack servers — slim units that slide into a rack; space-efficient for data centers
  • Blade servers — ultra-thin modules that share a common chassis for power and networking
• KVM switch (Keyboard-Video-Mouse) — allows one keyboard, monitor, and mouse to control many servers. Critical when the network is down and remote access is unavailable.
• Cloud computing — remote servers and data centers managed by a third party (e.g., AWS, Azure, GCP). The company rents compute resources instead of owning physical hardware.

Sys Admin Responsibilities

• Policy & security — in small companies the sys admin often sets organizational policies and owns computer security decisions end-to-end.
• Managing services — installing, configuring, updating, patching, and maintaining servers and the services running on them.
• Hardware lifecycle — four-stage process every piece of hardware goes through:
  1. Procurement — purchasing hardware, often in bulk from a vendor
  2. Deployment — setting up and configuring the hardware for use
  3. Maintenance — routine updates, patches, and repairs
  4. Retirement — decommissioning, wiping data, disposing or recycling
• Batch updates — rolling out security patches to many machines at once during scheduled maintenance windows.
• Vendor lifecycle — understanding manufacturer support windows so you know when hardware or software reaches end-of-life and bulk replacements need to be planned.
• Troubleshooting — reading logs is step 1; gathering information from the user (customer service) is step 2. The goal is to reproduce and isolate the problem before acting.

Applying Changes Safely

• Admin privilege mindset — admin rights give you enormous power. Key rules: respect privacy, think before you type, minimize time spent in elevated accounts, document everything.
• Session logging — record exactly what you did:
  • Linux: script session.log — captures the session in ANSI format; exit with exit
  • Windows PowerShell: Start-Transcript / Stop-Transcript — plain-text output
• Rollback plan — before changing anything, know how to undo it. Document the rollback steps as part of the change record.
• IT Change Management — a standardized process for planning, communicating, and implementing technical changes. A change record typically documents:
  • Person / team responsible
  • Priority and urgency
  • Description and purpose of the change
  • Scope and systems affected
  • Date, time, and expected duration
  • Rollback / backout plan
  • Risk level and anticipated impact
  • Resources and training needed
• Production vs test environment — never test in production. Production is the live environment end users rely on. Use a test environment (usually a VM mirroring production settings) to validate changes before rolling them out.
• Reproduction case — when troubleshooting, document: (1) the steps taken that led to the issue, (2) the unexpected result, (3) the expected result. Always reproduce on a test instance, not production.

IT Infrastructure Overview

• IT infrastructure has four main pillars: physical (hardware), network, software, and directory services.
• Cloud delivery models — what the provider manages for you:
  • IaaS (Infrastructure as a Service) — raw compute, storage, networking (e.g., Amazon EC2, Linode, Azure VMs)
  • NaaS (Network as a Service) — networking infrastructure delivered over the cloud
  • SaaS (Software as a Service) — fully managed applications (e.g., Gmail, Dropbox)
  • PaaS (Platform as a Service) — runtime environment for developers (e.g., Heroku, Google App Engine)
  • DaaS (Directory as a Service) — cloud-hosted directory (e.g., Azure AD, JumpCloud)

Physical Infrastructure: Servers & Remote Access

• Servers run their own specific operating systems (often Linux or Windows Server).
• Dedicated hardware — one service per physical machine. Better performance, but costly and underutilizes resources.
• Virtualized instances — multiple services share one physical machine via a hypervisor. Maximizes resource usage and allows live migration during hardware maintenance.
• OpenSSH — the most popular remote access tool on Linux. Requires an SSH client on your machine and an SSH server on the target.
  • Install: sudo apt-get install openssh-client / openssh-server
  • Connect: ssh user@ipaddress

Network Services

OS Imaging & Mass Deployment

• When a company is growing fast, manually setting up each new computer one-by-one is too slow. The solution is disk imaging — create one perfectly configured machine (the golden image), capture a snapshot of its entire disk, then flash that snapshot onto any number of new machines in minutes.
• A golden image (also called a base image or reference image) contains:
  • The operating system, fully installed and activated
  • Standard company software (Office, antivirus, VPN client, etc.)
  • Company-wide settings and security baselines
  • Any drivers needed for your hardware fleet
• Once a machine is imaged, it is domain-joined and receives any remaining per-user settings via Group Policy (GPO) automatically at login.

How PXE Boot Powers Mass Deployment

• PXE (Preboot eXecution Environment) — a machine boots from the network instead of its local disk. It contacts a DHCP server which tells it where to find a TFTP server, then downloads a boot loader and OS image over the network. No USB drive or DVD needed.
• The full flow for flashing a new computer:
  1. Technician plugs the machine into the network and powers it on.
  2. Machine sends a PXE broadcast — DHCP replies with the TFTP server address.
  3. Machine downloads a lightweight boot loader via TFTP.
  4. Boot loader pulls the golden image from the deployment server.
  5. Image is written to the local disk. Machine reboots into a fully configured OS.
• TFTP (Trivial FTP) is used here because it is extremely simple and requires no authentication — ideal for bare-metal machines that have no OS or credentials yet.

Deployment Tools

Unattended Installs & Answer Files

• An unattended install automates every prompt that would normally require a human to click through during OS setup — language, timezone, disk partitioning, product key, admin password, and more.
• Windows — uses an XML answer file called unattend.xml (created with the Windows System Image Manager tool). MDT reads this file during deployment and answers every setup question automatically.
• Linux — uses a kickstart file (Red Hat / CentOS) or a preseed file (Debian / Ubuntu). The file is referenced by the PXE boot loader and drives a fully automated install.
• Combined with PXE, an unattended install means a technician can rack a machine, plug in a network cable, and walk away — the machine configures itself end-to-end.

Managing System Services (Daemons)

• A daemon is a background process (service) that runs continuously. Daemons have one or more config files that control their behavior.
• Services are usually configured to start automatically at boot and on restart.
• Config files for installed services live in the /etc directory on Linux.

Configuring Network Services with Dnsmasq

• Dnsmasq is a lightweight package that provides DNS, DHCP, TFTP, and PXE services in a single program — ideal for small networks or lab environments.
• Install: sudo apt install dnsmasq — comes with sensible defaults and starts immediately.

• Dnsmasq caches DNS responses, speeding up repeat queries.
• NXDomain — the DNS response when a domain does not exist.
• DHCP client and server typically run on separate machines. The client uses dhclient to request an IP lease; the server uses dhcp.conf to define address ranges and options.

Troubleshooting Network Services

• Can't resolve a domain name? Work through this sequence:
  1. ping 8.8.8.8 — ping a known IP (like Google's DNS) to confirm basic connectivity
  2. nslookup <domain> — check DNS resolution; returns the name server and IP for the domain
  3. Compare the returned IP against what you expect — a mismatch points to a DNS misconfiguration
• If ping to an IP succeeds but DNS resolution fails, the problem is DNS-specific, not network-wide.

Communication Services

• IRC (Internet Relay Chat) — free, open-source chat protocol. A self-hosted alternative to Slack or Teams for internal team communication.
• XMPP (Extensible Messaging and Presence Protocol) — popular open standard for instant messaging. Used under the hood by many chat systems.
• For email you can either run your own mail server or pay an email service provider (Google Workspace, Microsoft 365). Self-hosting gives full control; a provider reduces operational burden.
• DNS records for email:
  • A record — maps a hostname to an IP address
  • MX record — Mail Exchange; tells the internet which server accepts email for a domain

• Spam mitigation — three DNS-based mechanisms that verify the sender is who they claim to be:
  • SPF (Sender Policy Framework) — lists which IP addresses are authorized to send email for your domain
  • DKIM (DomainKeys Identified Mail) — cryptographically signs outgoing messages so receivers can verify they weren't tampered with
  • DMARC (Domain-Based Message Authentication, Reporting & Conformance) — ties SPF and DKIM together and tells receivers what to do with mail that fails (quarantine, reject, or do nothing)
• User Productivity Services — business licensing for productivity suites (Office 365, Google Workspace) is separate from personal licensing. Enterprise versions include admin controls, compliance tooling, and user management.

Security Services — HTTPS & TLS

• Any service you run must guarantee to users that their information is handled securely — especially anything accepting credentials or personal data.
• HTTPS — secure version of HTTP. Encrypts all communication between the browser and the web server so data cannot be read or tampered with in transit.
• TLS (Transport Layer Security) — the cryptographic protocol that powers HTTPS. It is the current standard for securing network communication.
• SSL (Secure Sockets Layer) — the predecessor to TLS. SSL v3.0 was effectively renamed TLS v1.0. SSL is deprecated and should never be used for new systems.
• To enable HTTPS you need a TLS certificate issued by a trusted Certificate Authority (CA). The CA vouches that the certificate belongs to the domain it claims to represent. Let's Encrypt is a free, automated CA widely used for this purpose.

File Services

• Choosing the right file system or sharing protocol depends on who needs to access the files and from what OS.

• On Linux, NFS clients mount a remote share using the host path: e.g., mount host:/nfs /mnt/shared.
• Mobile device file sync — always sync mobile data to a cloud server or local IT infrastructure. If the device is lost, data is not lost with it.

Print Services

• Large organizations need a print server to manage a fleet of printers centrally — handling queues, drivers, and permissions from one place.
• CUPS (Common Unix Printing System) — the built-in Linux print server. Most modern OSes ship with a print server included.
• When provisioning new computers, have printer drivers and default settings (orientation, quality, tray, duplex) pre-staged so users can print immediately.
• Cloud print management services let both users and admins manage printers through a web interface — useful for distributed offices.

Platform Services — Web Servers & Databases

• Platform services give developers an environment to build and deploy software without managing the underlying hardware or OS — this is the PaaS model.
• A web server is a program (or the machine running it) that listens for HTTP requests and responds with the requested files or data.

• Load balancer — sits in front of multiple servers and distributes incoming traffic evenly. Can be hardware or software. Prevents any single server from being overwhelmed and enables horizontal scaling.
• Databases — store, retrieve, and manage structured data. The two most popular open-source relational databases are MySQL and PostgreSQL.

Troubleshooting Platform Services — HTTP Status Codes

• When something goes wrong with a web service, HTTP status codes are your first diagnostic signal — they tell you immediately whether the problem is on the client side or the server side.

Managing Cloud Services

• SaaS — the software is fully pre-configured; you choose from a small set of admin options. You don't manage the underlying infrastructure at all.
• IaaS — you configure and manage your services on rented hardware. Start small; scale as you understand your actual usage patterns.
• Cloud deployment models:
  • Public — infrastructure owned and operated by a provider, shared across many customers (AWS, GCP, Azure)
  • Private — cloud infrastructure run entirely by your own company; more control, higher cost
  • Hybrid — a mix of public and private; sensitive workloads stay on-premises, burst capacity moves to public cloud
• Regions — cloud providers organize their data centers into geographic regions. Choose regions close to your users to minimize latency; use multiple regions for redundancy.
• Load balancer — in a cloud context, ensures each VM instance receives a balanced share of queries as traffic scales up.
• Autoscaling — automatically increases or reduces VM capacity in response to actual demand. Ideal for workloads with variable traffic (seasonal spikes, marketing events).
• Cloud infrastructure avoids large upfront capital investment — you pay for what you use. This makes it especially cost-effective when demand varies significantly throughout the year.

Introduction to Directory Services

• A directory server is a lookup service that maps network resources (users, groups, devices, phone numbers) to their network addresses — like a phone book for your IT infrastructure.
• Replication — directory data is copied across multiple physically distributed servers but still appears as one unified system for queries. Benefits:
  • No single point of failure — if one replica goes down, others serve requests
  • Reduced latency — clients query the geographically nearest replica
• Data is organized hierarchically like a filesystem: company domain → department OUs → individual objects. Organizational Units (OUs) can contain objects or other OUs.
• The sysadmin owns setup, configuration, and ongoing maintenance of the directory.
• Standard directory protocols come from the X.500 family: DAP, DSP, DISP, DOP. These are complex and heavyweight.
• LDAP (Lightweight Directory Access Protocol) is the simplified, practical alternative used everywhere today. It gives clients network access to directory data.
• Two dominant implementations: Microsoft Active Directory and OpenLDAP (open-source).

Centralized Management & AAA

• Centralized management — a single service that issues instructions and enforces policy across the entire IT infrastructure. Without it, each machine is an island with its own local accounts.
• Directory services provide centralized AAA:
  • Authentication — prove who you are
  • Authorization — determine what you're allowed to do
  • Accounting — log what you actually did
• RBAC (Role-Based Access Control) — permissions are assigned to roles, not individuals. A user's access is determined by their role in the org.
• When someone changes roles, leaves, or joins — update their group membership. No need to touch individual machine accounts.
• Centralized policy can be as simple as logon scripts that map drives or printers when a user signs in.

LDAP — Protocol & Notation

• LDAP is the wire protocol used to read and write data in a directory service. Both Active Directory and OpenLDAP speak LDAP.
• Every directory entry has a distinguished name (DN) — a unique, comma-separated path that identifies its location in the hierarchy:

• Common LDAP operations: add entry, modify entry, delete entry, search.
• The Bind operation authenticates a client to the directory server before it can perform operations. Three authentication modes:
  • Anonymous — no credentials; read-only public data
  • Simple — username and password in plain text (use only over TLS)
  • SASL (Simple Authentication and Security Layer) — pluggable auth; typically uses TLS and supports Kerberos
• Kerberos — a network authentication protocol used to prove identity without sending passwords over the wire. Widely used with Active Directory.

Active Directory (AD)

• Active Directory is Microsoft's native directory service for Windows environments. It speaks LDAP, so it can interact with non-Windows hosts as well.
• ADAC (Active Directory Administrative Center) is the primary GUI for managing AD objects.
• Key structural concepts:
  • Domain — the basic administrative boundary; a collection of computers, users, and resources
  • Forest — one or more domains that share a schema and trust each other
  • Organizational Unit (OU) — a folder-like container inside a domain; used to organize objects and delegate admin control
  • Domain Controller (DC) — the server running AD; it authenticates logins and enforces policy
  • FSMO (Flexible Single Master Operation) — certain AD tasks require one authoritative DC to avoid conflicts (PDC Emulator, RID Master, etc.)
  • SAM (Security Account Manager) — stores usernames and hashed passwords

• AD has three group scopes — choose based on what you're grouping and where it's used:
  • Domain Local — assign permissions to a specific resource (e.g., a shared folder)
  • Global — group accounts by role within a domain (e.g., "RnD" global group contains "Researchers" sub-group)
  • Universal — group global roles across an entire forest
• Security groups can contain user accounts, computer accounts, or other security groups — used to grant access to resources.
• Distribution groups are email-only — they cannot be used to assign permissions.
• Admins should never know a user's password. Only reset when you've absolutely verified the request is legitimate. One person, one authenticator.
• Tip: ADAC shells out to PowerShell. Watch the command history pane to see the exact commands it runs — then script those for bulk operations.

Group Policy Objects (GPOs)

• A GPO is a set of policies and preferences that can be applied to a group of AD objects (users, computers, or both).
• GPOs must be linked to a domain, site, or OU to take effect — creating a GPO alone does nothing.
• Each GPO has two sections:
  • Computer Configuration — applied when the machine boots
  • User Configuration — applied when the user logs on
• Policies — settings that are re-enforced every few minutes; users cannot override them.
• Preferences — settings that act as templates; users can change them after the fact.
• GPO data is stored in the SYSVOL folder, which is replicated to all DCs.
• The Windows Registry is the hierarchical database where GPO policy settings ultimately land on the client machine.

GPO Processing Order (LSDOU)

• GPOs are applied in this order — later wins when there are conflicts:

1. Local GPO — settings stored directly on the machine
2. Site-linked GPOs
3. Domain-linked GPOs
4. OU-linked GPOs — most specific container is applied last (and wins)

• Within an OU, each GPO has a link order. Higher link-order number = applied first = lowest precedence. The GPO applied last wins.
• An upstream GPO can be set to Enforced to prevent child OUs from overriding it.
• An OU can be set to Block Inheritance to ignore GPOs from parent containers (unless a parent GPO is Enforced).

GPO Tooling & Troubleshooting

GPO Troubleshooting Checklist

1. Check the GPO scope — is it linked to the right OU/domain?
2. Check security filtering — does the target user/computer have Read and Apply permission?
3. Verify Read and Apply permissions on the GPO
4. Check Group Policy delegation settings
5. Is the relevant config section (Computer or User) enabled?
6. Confirm the LSDOU processing order — is a higher-priority GPO overriding?
7. Ensure the GPO link is enabled (not just created)
8. Is an upstream GPO set to Enforced, blocking your override?
9. Is the affected OU set to Block Inheritance?
10. Is Loopback processing enabled? (changes how User Config applies to shared machines)
11. Check WMI filters — an unexpected filter may exclude the target
12. Verify your expectations match the GPO setting's actual behavior

OpenLDAP

• OpenLDAP is the most popular open-source LDAP implementation. It works on any OS and operates similarly to Active Directory's LDAP layer.
• Two management interfaces:
  • phpLDAPadmin — web-based GUI; easiest for browsing and editing entries
  • CLI tools — scriptable; required for automation

• LDIF (LDAP Data Interchange Format) — plain-text files that describe directory entries or changes. The workflow is: write an LDIF file → run the appropriate ldap* command. That's it.

dn: uid=jane,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
cn: Jane Smith
sn: Smith
uid: jane
mail: jane@example.com

Planning for Data Recovery

• Data recovery — the process of restoring data after an unexpected event involving data loss or corruption. The main objective is to resume normal operations as soon as possible.
• The most important technique is backing up regularly. Restoring from backup is always the best solution when one exists.
• Backup systems should prioritize important/essential data and account for future growth.
• On-site backups — data is physically nearby, low bandwidth to access. Risk: a local disaster destroys all copies.
• Off-site backups — data is safer across multiple locations. Requires security, encryption (TLS preferred), and sufficient bandwidth. Best practice is to maintain both on-site and off-site backups.
• Long-term archival — magnetic tape is the standard medium: inexpensive and long-lasting.
• Hard disks will fail eventually — redundancy and regular backups are non-negotiable.
• User backups — can be handled via SaaS solutions (Dropbox, iCloud, etc.) that are easy for users to configure.
• OS built-in tools:
  • macOS → Time Machine
  • Windows → Backup & Restore
  • Linux → rsync — a file-transfer utility designed to efficiently copy and sync files, transferring only changed data
• Regular testing — backups must be tested regularly. Restoration procedures must be documented and exercised at least once a year (disaster recovery testing).

Recovery Objectives

Backup Types

• Full backup — copies all data every time. Simple to restore but slow and storage-heavy for largely-static data.
• Differential backup — copies everything changed since the last full backup. Saves storage vs. repeated full backups; restore requires last full + latest differential.
• Incremental backup — copies only data changed since the last backup of any type. Most storage-efficient; restore requires last full + every incremental in order.
• Best practice: infrequent full backups combined with more frequent differential or incremental backups.
• Compression — backups can be compressed to save space, but not all data types compress well. Compressed files must be decompressed before restoration.

RAID Levels

Disaster Recovery Plans

• A disaster recovery plan (DRP) is a collection of documented procedures for reacting to an emergency or disaster scenario, with the goal of minimizing disruption to business operations.
• A DRP covers three categories of measures:
  1. Preventative — actions taken to reduce the likelihood of a disaster
  2. Detection — monitoring and alerting systems that identify an incident quickly
  3. Corrective / Recovery — steps to restore operations after an incident
• Risk assessment — identify high-risk systems; pay special attention to systems that lack redundancy.
• Determine which data/systems are highest priority and have a data recovery plan ready for each.
• Verify that all operational documentation is current and accessible.
• Define and test detection and alert measures so the team knows about an incident immediately.
• Document and test recovery procedures in detail.

Post-Mortems

• A post-mortem is created after an incident, outage, or event — or at the end of a project — to analyze what happened and how to improve.
• Purpose: to learn, not to punish or shame. Understanding is the goal.
• Standard post-mortem structure:
  1. Brief summary — what the incident was, how long it lasted, what the impact was, and how it was resolved
  2. Detailed timeline — every significant event and every attempt to fix the issue, with exact dates, times, and time zones
  3. Root cause — the underlying reason the incident occurred and what can be learned from it
  4. Resolution & recovery — detailed account of the steps taken to fix the problem and restore service
  5. Action items — concrete steps to prevent the same scenario in the future
• Highlight both what went wrong and what went well — cover everything.
• Post-mortems also follow the end of a project to capture lessons learned even when nothing went wrong.

How to use these case studies

• Read the company snapshot and the problems identified — then try to work out your own approach before revealing mine.
• The analysis is my own, written during the course. It isn't a model answer or AI-generated — it's how I thought through each scenario at the time. Your approach is most likely different so don't take mine as gospel! Lots of different approaches work!
• Each point is tagged § Section N to show which part of the course I was drawing on.

Scenario 1 — Network Funtime Company

Problems identified

• Procurement is reactive and inefficient — one laptop at a time at the cheapest price, no vendor relationship, no bulk deal
• Every employee has different hardware — impossible to standardize support, drivers, or images
• No asset inventory — no way to audit what the company owns or track machine lifecycle
• Employees self-installing software risks bloatware, misconfiguration, and security holes
• HR is doing IT work and burning hours per hire — not scalable and not their job
• No password policy and no recovery path — lost credential = wipe and start over
• Cloud service access handed out ad-hoc with no central directory or SSO

Scenario 2 — W.D. Widgets

Problems identified

• Manual machine setup with long app installs will not scale to hundreds of new hires — IT will be a bottleneck
• File server has no access controls — any employee can permanently delete shared customer data
• Zero backups — a single failure wipes everything, with no recovery path
• Email-only IT support will be overwhelmed as headcount grows; no prioritization or tracking
• In-house everything at rapid scale means IT team needs to grow or infrastructure will buckle

Scenario 3 — Dewgood (Non-Profit)

Problems identified

• Day-of retail procurement is the most expensive and least reliable option — whatever's in stock wins
• Departing employees stay active in AD — security risk and directory clutter
• Backup disk goes home with IT every night — one lost or damaged drive means data loss
• Single server for files, email, and website — no redundancy; one crash takes everything offline
• Static website going down on shared hardware is an unmonitored single point of failure
• Ticketing system exists but has zero adoption — IT support is informal and untracked