Syllabus: Unix Tools: Data, Software and Production Engineering
Index
- 1. Course overview
- 2. Learning objectives
- 3. What we expect from you
- 4. What you can expect from us/the course team
- 5. Course structure
- 6. Resources, Tools & Browsers
- 7. Assessment & Certificate
- 8.
Licence
1. Course overview
Processing information is the hallmark of all modern organizations, which are increasingly digital: absorbing, processing and generating information is a key element of their business. Being able to interact flexibly and efficiently with the underlying data and software systems is an indispensable skill. Knowledge of the Unix shell and its command-line tools boosts the effectiveness and productivity of software developers, IT professionals, and data analysts.
The Unix tools were designed, written, actively used and refined by the team that defined the modern computing landscape. They allow the performance of almost any imaginable computing task quickly and efficiently by judiciously combining key powerful concepts. The power of Unix tools for exploring, prototyping and implementing big data processing workflows, and software engineering tasks remains unmatched. Unix tools, running on hardware ranging from tiny IoT platforms to supercomputers, uniquely allow an interactive, explorative programming style, which is ideal for the efficient solution of many of the engineering and business analytics problems that we face every day.
The course is uniquely based on carefully-selected, interactive walk-through examples that demonstrate how each command operates in practice. The examples that we use involve problems that engineers and analysts face every day.
2. Learning objectives
At the end of this course, you will be able to:
- Enter and combine commands in the Unix command line
- Use files, data pipelines, variables and control structures
- Select the most useful tools and commands for fetching, selecting, generating, processing, summarizing and reporting data
- Obtain data from databases, cloud-based hosts, version control systems, object files, archives and your desktop files
- Accomplish diverse processing tasks by putting together suitable commands and configuring their execution parameters
3. What we expect from you
As an online student, we expect you to be an active participant in this course by contributing to a positive atmosphere. We want you to question, share and help others by engaging in meaningful discussions.
Regarding deadlines, we expect you to keep on track in order to benefit from learning within a community. This course is meant to be a place where you learn with and from others. In this sense, we would like you to experience collaboration and peer-feedback, so please make sure you follow along with other participants in order to enrich the overall learning experience.
You are expected to follow forum and collaboration guidelines. Respect the course policies, academic integrity and most importantly your fellow students.
4. What you can expect from us / the course team
The moderator will guide you throughout the course, launching the weekly content, promoting and engaging in discussions, and providing feedback regarding your performance after each week. Guidance and support will happen on a regular basis.
Response Time: We will try to respond to all your questions and posts within 24-48 hours. If this not possible for any reason, we will let you know.
5. Course structure
The course is organized in six modules. We start by introducing the key ideas and advantages of Unix tools. We then build many one-liners around the tools that follow a sequence of fetching, selection, processing, summarization and reporting. We demonstrate the tools for each phase: the plumbing that joins the parts into a whole, more specialized commands for handling software development and system administration tasks, sound and images and even graph-structures. We end with a discussion of common use patterns to follow and anti-patterns to avoid.
A brief summary of each unit is presented below.
1. Foundations (released on March 17th, 2020)
The first module covers the fundamentals: getting to the command line; introducing the command-line interface; performing input and output redirection; handling files and directories; providing command-line arguments; grouping commands; scripting; execution control; and an overview of the data processing flow covered by the next units.
2. Data Fetching (released on March 24th, 2020)
The second module covers how you can obtain data from diverse sources: remote services, archives, remote hosts, the file system, version control systems, compiled code, graphical desktop systems, system administration entities, and even generators.
3. Selection (released on March 31st, 2020)
The third module covers selecting data with regular expressions and the grep family of tools and other relevant ones.
4. Processing (released on March 31st, 2020)
The fourth unit covers processing through sorting and working with sorted data, the mini-languages of the sed and awk tools, obtaining file differences, testing, evaluating expressions, and dealing with characters, lines, graphs, images, and sound.
5. Reporting (released on April 14th, 2020)
The fifth unit covers reporting through summarizing, emailing, and text formatting.
6. Professionalism (released on April 14th, 2020)
Finally the last, sixth unit covers good shell practices and advice for improving your shell style.
Please note that the estimated workload for this course is between 4 and 6 hours weekly. Some of the topics are longer than others; bearing this in mind, the content of Topics 3 & 4 will be released on the same date but you will have two full weeks to complete them. The content of Topics 5 & 6 will also be open on the same date, and you will have until the end of the course (April 28th, 2020) to catch up on all assignments, if need be.
6. Resources, Tools & Browsers
All educational resources will be available in the course. They consist of short videos and mostly quizzes to support you in the completion of the course topics.
To follow the course, you will need to obtain a Unix command-line prompt. How you get to the Unix command line depends on the system you're using. Please watch the video Getting to the command line to learn about your options under Windows, macOS or the Ubuntu GNU/Linux distribution.
We support the following browsers: Chrome, Firefox and Safari.
7. Assessment & Certificate
Throughout the course you’ll be monitoring your progress through more than 110 knowledge checks. Knowledge checks are available to all course participants, regardless of having enrolled as audit or verified learners. They are ungraded, thus they don't count towards the course final score.
In addition, only learners in the verified track will have access to five progress assessments interspersed throughout the course. These will allow you to demonstrate in practice how you combine your newly acquired skills by completing about twenty carefully selected tasks modeled after real-life scenarios. Each progress assessment counts for 20% of the course final grade. In order to pass the course, you'll need a score of 60%.
Verified participants can check their score at any time under the course’s Progress page. Upgrade to a Verified Certificate also gives you:
- a certificate if you successfully completed the course;
- access to the archived course after the end date.
These certificates will indicate you have successfully completed the course, but will not include a specific grade. Certificates will be issued by edX under the name of DelftX, designating the institution from which the course originated.
Do you need financial assistance? EdX offers up to a 90% discount on our verified certificates to learners who cannot afford to pay full price. Check the edX support page for financial assistance.
Generating an ID verified certificate
Verified certificates will be issued a few days after the end of the course, to verified participants who successfully completed the course. Certificates can be downloaded from your Student Dashboard (look for the Download button next to the name of our course). An ID verified Certificate of Achievement is available for $50. You can Upgrade on your edX Dashboard to Verified during the course. Once produced, a certificate cannot be reissued, hence it is very important that you verify the way in which your name appears. Check that, in your edx.org account, your name is correctly spelled, since it will appear on the final certificate.
8. Licence
The course materials of this course are
If you choose to reuse or repost DelftX course materials you must give proper attribution. Please utilize the following citation and refer to this MOOC:
"[TITLE OF WORK –with hyperlink to material] by TU Delft / [LECTURER NAME -with hyperlink to lecturers page] is licensed under CC-BY-NC-SA 4.0. This material was created by or adapted from material posted on [TITLE MOOC- with link to MOOC start page].”
Or if it is a derivative please use
“This work [Your title] by [Your name] is a derivative of “TITLE OF WORK –with hyperlink to material] by TU Delft / [LECTURER NAME -with hyperlink to lecturers page] and (re)licensed under CC-BY-NC-SA 4.0. This material was created by or adapted from material posted on [TITLE MOOC- with link to MOOC page].”
