text:
trust,
empowerment, and collaboration."  These value manifest themselves in four
dimensions in Scrum:

* the roles of the people that participate in Scrum;
* the activities in which a team using Scrum will participate;
* the artifacts created to manage the process; and,
* the rules that bind the interactions of the roles, activiites, and artifacts.

## Scrum roles

When you work on a team using Scrum, the people on that team take on a role in
the Scrum process. The three roles are **product owner**, **Scrum master**, and
a member of the **development team**. It is best when no one person fills more
than one role. (Of course, many companies say, "We can customize Scrum to meet
our own special way of doing things, and our special way of doing things is to
be cheap and not spend money on humans to actually help out with product
development!") The product owner shoulders the responsibility for what will be
developed and the order in which the features of the software will be
implemented. The Scrum master gets to guide the team in creating and following
its own Scrum-based process. The members of the development team determine how
to deliver the features asked for by the product owner.

So, where is "manager" or "team lead"? Where is "Vice President" and "CEO? You
should note that **product owner**, **Scrum master**, and being a member of the
**development team** are _roles_ and not titles. Anyone can fit into those roles
as long as they live up to the expectations set by that role.

### The product owner

The product owner provides the leadership needed to define the product. That
person has the singlular authority to decide which features and functionality to
build and in which order to build them. The product owner must have a clear
vision of what the product the Scrum team is trying to build. Because this
person must define and communicate that vision, the product owner is responsible
for the overall success of the software (or other product) in development or
maintenance.

It doesn't matter if the focus for the software being built is something to be
sold to consumers (B2C software), sold to other businesses (B2B software), or
used solely within the company (internal software). The product owner has the
obligation to make sure the team performs the most valuable work possible. To
ensure that the team rapidly builds what the product owner wants, the product
owner actively collaborates with the Scrum master and development team. The
product owner must avail themselves for questions from the development team that
could arise at any time.

### The Scrum master

The Scrum master helps everyone understand and embrace the Scrum values and
principles, the practices and procedures. The Scrum master provides process
leadership. The Scrum master helps the Scrum team and the rest of the company
develop their own organization-specific, high-performance adaptation of Scrum.

As a facilitator of Scrum, the Scrum master helps the team continuously improve
it use of Scrum, allowing the team to focus on its product development
priorities. Preventing the team from getting distracted by outside interference
and by removing roadblocks that inhibit the team's productivity, the Scrum
master plays the pivotal role of the facilitator of team focus.

It is extremely important to note that the Scrum master is _not_ a project
manager or development manager.  Instead, the Scrum master acts as a _leader_, not a
manager.

### The development team

You may have heard of different types of people that fit into the software
development process: QA tester, database administrator, user interface designer,
user experience engineer, programmer, architect, and more. Scrum eschews any of
those and provides only the single term of "development team" to encapsulate and
acknowledge the diverse group of people that it takes to make any non-trivial
software product.

The development team organizes itself around the principles of Scrum to
determine the best way to practice Scrum. A development team typically ranges in
size from five to ten people. Together, the members of the team have the
necessary skills to produce well-designed and well-tested software. Some very
large companies practice Scrum; instead of having their hundreds of software
developers all on a single huge Scrum team, they will normally create groups of
five to ten people to make up small Scrum teams all working on parts of a larger
product.

## Scrum activities. Scrum artifacts.

The activities that make up the performances of the different roles of a Scrum
team are simple to list. They are a cycle that gets performed over and over
again.

1. The product owner has a vision of what needs to be created.
2. The product owner takes that vision and breaks it down into a list of
   features yet to be implemented called the **backlog**.
3. The product owner manages the backlog through a process called **grooming**
   which leaves the list in a prioritized order.
4. The development process begins with an activity named **sprint planning**
   which culminates in the team forecasting how many features they will be able
   to complete within a predetermined amount of time.
5. Once planning ends, the Scrum team attempts to complete the features that
   that they forecasted they could complete. Each day during sprint execution,
   the team members help coordinate their work through am inspection,
   synchronization, and adaptive planning activity known as the **daily scrum**.
   (Notice the lowercase "s" in "daily scrum".)
6. The team completes the Sprint by performing two more inspection and
   adaptation activities: the **show and tell** and the **sprint
   retrospective**. The outcome of these activities can alter the way the team
   decides to complete their work in the future.

The time that it takes to perform steps 4, 5, and 6 is the predetermined amount
of time called a **sprint**. A sprint normally lasts from one week to one month
depending on how the team has implemented Scrum.

### The product backlog

All Agile software development methodologies try to do the most valuable or the
most complex work first. The product owner, with input from the rest of the
Scrum team (and bosses), determines and manages this sequnce of work by
communicating it to the Scrum master and development team through the
prioritized list known as the _product backlog_. When creating new products,
usually the product owner fills the backlog with features that are required to
meet the vision of the product for it to go to market. For on-going product
development (maintenance mode), the product backlog will usually contain new
features combined with changes to existing features, bugs that need repairing,
fixes to technical debt, and more,

The product owner gathers the priorities of external and internal stakeholders
to define the items in the product backlog. Based on those priorities, the
product owner orders the items in the product backlog so that the high-value
items appear at the top of the product backlog. The product owner grooms the
backlog in that manner as often as necessary to make sure that the highest
priority items are always at the top of the list.

### Sprints

In Scrum, work is performed in iterations or cycles of up to a month called
sprints. The work completed in each sprint should create something of tangible
value for the people that use the software.

Sprints are **timeboxed** so they always have a fixed start and end date. They
are usually all the same duration. A new sprint immediately follows the
completion of an old sprint. As a rule, the team does do not change the team
members or what they work on during a sprint; however, business needs sometimes
make it impossible to follow that rule.

### Sprint planning

A product backlog may represent many weeks or months of work, which is much more
than can be completed in a single, short sprint. To determine the most important
subset of product backlog items to build in the next sprint, the product owner,
development team, and Scrum master perform **sprint planning**.

During sprint planning, the product owner and development team agree on a sprint
goal that defines what the upcoming sprint is supposed to achieve. Using this
goal, the development team reviews the product backlog and determines the
high-priority items that the team can realistically accomplish in the upcoming
sprint while working at a sustainable pace. It is important to note that the
team agrees on what it believes to be its own sustainable pace. Some teams can
work longer than others. Some teams have maintenance commitments. Each team has
different demands on it.

To acquire confidence in what it can get done, many development teams break down
each targeted feature into a set of tasks. The collection of these tasks, along
with their associated product backlog items, forms a second backlog called the
**sprint backlog**.

The development team then provides an estimate (typically in hours) of the
effort required to complete each _task_. Breaking product backlog items into
tasks is a form of design and just-in-time planning for how to get the features
done.

## Running the sprint

Once the Scrum team finishes sprint planning and determines the features that it
will complete in the next sprint, the development team, guided by the Scrum
master’s coaching, performs all of the task-level work necessary to get the
features done. “Done” means that the software developers have performed tasks to
ensure the highest quality of software, and that the product owner has approved
each feature's completion.

Nobody tells the development team in what order or how to do the task-level work
in the sprint backlog. Instead, team members define their own task-level work
and then self-organize in any manner they feel is best for achieving the sprint
goal.

### Daily scrum

Each day of the sprint, ideally at the same time, the development team members
hold a short (15 minutes or less) **daily scrum**. This activity is often
referred to as the **daily stand-up** because of the common practice of everyone
standing up during the meeting to help promote brevity.

A common approach to performing the daily scrum has each team member taking
turns answering three questions for the benefit of the other team members:

* What did I accomplish yesterday?
* What do I hope to accomplish today?
* What obstacles or impediments prevent me from making progress?

By answering these questions, everyone understands the big picture of what is
occurring, how they are progressing toward the sprint goal, any modifications
they want to make to their plans for the upcoming day’s work, and what issues
need to be addressed. The daily scrum is essential for helping the development
team manage the fast, flexible flow of work within a sprint.

The daily scrum is _not_ a problem-solving activity. Rather, many teams decide
to talk about problems _after_ the daily scrum. They will form small groups of
interested people. The daily scrum also is not a traditional status meeting. A
daily scrum, however, can be useful to communicate the status of sprint backlog
items among the development team members. Mainly, the daily scrum is an
inspection, synchronization, and adaptive daily planning activity that helps a
self-organizing team do its job better.

### Done

In Scrum, the sprint results should be "shippable" software, meaning that
whatever the Scrum team agreed to do is really done according to its agreed-upon
definition of done. This definition specifies the degree of confidence that the
work completed is of good quality and is potentially shippable. For example,
when developing software, a bare-minimum definition of done should yield a
complete slice of product functionality that is designed, built, integrated,
tested, and documented.

A holistic definition of done enables the business to decide each sprint if
it wants to make the software available to its customers, internal or external.

As a practical matter, over time some teams may vary the definition of done. For
example, in the early stages of game development, having features that are
potentially shippable might not be economically feasible or desirable (given
the exploratory nature of early game development). In these situations, an
appropriate definition of done might be a slice of product functionality that is
sufficiently functional and usable to generate feedback that enables the team to
decide what work should be done next or how to do it.

### Show and tell

Show and tell gives the opportunity for everyone on the team to see what has
been created. In the case of many Scrum teams working in concert, this provides
a way for teams to see the work performed by other team. Critical to this
activity is the conversation that takes place among its participants, which
include the Scrum team, stakeholders, sponsors, customers, and interested
members of other teams. The conversation is focused on reviewing the just-com-
pleted features in the context of the overall development effort. Everyone in
attendance gets clear visibility into what is occurring and has an opportunity
to help guide the forthcoming development to ensure that the most
business-appropriate solution is created.

A successful review results in information flowing in both directions. The
people who aren’t on the Scrum team get to sync up on the development effort and
help guide its direction. At the same time, the Scrum team members gain a deeper
appreciation for the business and marketing side of their product by getting
frequent feedback on the growth of the product. The show and tell therefore
represents a scheduled opportunity to inspect and adapt the product.

### Sprint retrospective

The other activity at the end of the sprint is the **sprint retrospective**. It
frequently occurs after the show and tell and before the next sprint planning.

Whereas the show and tell is a time to inspect and adapt the product, the sprint
retrospective is an opportunity to inspect and adapt the process. During the
sprint retrospective the development team, Scrum master, and product owner come
together to discuss what is and is not working with all of the development
practices. The focus is on the continuous process improvement necessary to help
a good Scrum team become great. At the end of a sprint retrospective the Scrum
team should have identified and committed to a practical number of process
improvement actions, actions that they will enact during the next sprint.

After the sprint retrospective is completed, the whole cycle is repeated again—
starting with the next sprint-planning session, held to determine the current
highest- value set of work for the team to focus on.

## What you learned

You learned that Scrum is a framework that teams can use to build a sustainable
product-development process. The framework defines

* The product owner as the person that directs the highest priority development
  of the product
* The Scrum master that helps the team follow its process and be most efficient
* The development team that completes the work that the product owner specifies
* The sprint, some amount of time during which the Scrum process is run
* Sprint planning which provides the road map for what to accomplish during a
  sprint
* The daily scrum, a meeting that the Scrum team attends to communicate its
  activities and blockers
* The show and tell which allows a larger group visibility into the products of
  the sprint
* The retrospective, which allows the team to determine process improvements
* Backlog grooming performed by the product owner to ensure that the backlog of
  stories is well ordered and defined

That's the framework of Scrum, the most popular Agile software development
methodology in use today.

________________________________________________________________________________
# Let's Talk About Sprints

Scrum organizes work in iterations or cycles of up to a calendar month called
sprints. This reading provides a more detailed description of a sprint.
It then discusses several key characteristics of sprints

* That they are timeboxed;
* That they have a short and consistent duration
* That they have an unalterable goal; and,
* That they can only conclude using the team’s definition of done.

Although each team or company will have its own unique implementation of Scrum,
these sprint characteristics, with few exceptions, are meant to apply to every
sprint and every team.

## Put your sprint in a pretty timebox

Sprints are rooted in the concept of **timeboxing**, a time-management technique
that helps organize the performance of work and manage scope. Each sprint takes
place in a time frame with specific start and end dates, called a **timebox**.
Inside this timebox, the team is expected to work at a sustainable pace to
complete a chosen set of work that aligns with a sprint goal. Timeboxing is
important for several reasons.

A reason to timebox is to _establish work-in-progress (WIP) limits_. WIP
represents an inventory of work that is started but not yet finished. Failing to
properly manage it can have serious economic consequences. Because the team will
plan to work on only those items that it believes it can start and finish within
the sprint, timeboxing establishes a WIP limit each sprint.

Timeboxing forces the team to _prioritize_ and perform the small amount of work
that matters most. This sharpens the team's focus on getting something valuable
done quickly.

Timeboxing also helps us _demonstrate relevant progress_ by completing and
validating important pieces of work by a known date (the end of the sprint).
This type of progress reduces  risk by shifting the focus away from unreliable
forms of progress reporting, such as conformance to a project plan. Timeboxing
also helps to demonstrate progress against big features that require more than
one timebox to complete. Completing some work toward those features ensures that
valuable, measurable progress is being made each sprint. It also helps the
stakeholders and team learn exactly what remains to be done to deliver the
entire feature.

Timeboxing helps _avoid unnecessary perfectionism_. At one time or another
people spend too much time trying to get something “perfect” when “good enough”
would suffice. Timeboxing forces an end to potentially unbounded work by
establishing a fixed end date for the sprint by which a good solution must be
done.

Timeboxing also _motivates closure_. Teams can focus on getting things done when
they have a known end date. The fact that the end of the sprint brings with it a
hard deadline encourages team members to diligently apply themselves to complete
the work on time. Without a known end date, there is less of a sense of urgency
to complete the job.

Timeboxing _improves predictability_. Although we can’t predict with great
certainty exactly the work we will complete a year from now, it is completely
reasonable to expect that we can predict the work we can complete in the next
short sprint.

## Take advantage of this short-term deal!

Short-duration sprints make it _easier to plan_. It is easier to plan a few
weeks’ worth of work than six months’ worth of work. Also, planning on such
short time spans requires far less effort and is far more accurate than
longer-term planning.

Short-duration sprints _generate fast feedback_. During each sprint, the team
creates working software. Then, they have the opportunity to inspect and adapt
what was built and how they built it. This fast feedback enables the team to
quickly prune unfavorable product paths or development approaches before those
choices can compound a bad decision with many follow-on decisions that are
coupled to the bad decision. Fast feedback also allows us to more quickly
uncover and exploit time-sensitive emergent opportunities.

Short-duration sprints not only improve the economics via fast feedback; they
also allow for _early and more frequent deliverables_. As a result, the company
can have the opportunity to generate revenue sooner, improving the overall
return on investment.

Short-duration sprints also _bound error_. How wrong can a team get in a
two-week sprint? Even if the team does everything wrong, the group has lost only
two weeks. Scrum insists on short-duration sprints because they provide frequent
coordination and feedback. That way, if the team gets something wrong, then
they get something wrong in only a small way.

Short-duration sprints also provide _multiple, meaningful checkpoints_. One
valued aspect of sequential projects is a well-defined set of "milestones".
These milestones provide managers with expectations about what should be
delivered by a certain date. If things don't happen by that date, it allows the
team to decide if the project should continue. Although potentially useful from
a governance perspective, these milestones give an unreliable indication of the
true status of a project.

Scrum provides managers, stakeholders, product owners, and others with many more
checkpoints than they would have with sequential projects. At the end of each
short sprint, there is a meaningful checkpoint (the show and tell) that allows
everyone to base decisions on demonstrable, working features. People are better
able to deal with a complex environment when they have more actionable
checkpoint opportunities to inspect and adapt.

## The duration remains the same

As a rule, on a given development effort, a team should pick a consistent
duration for its sprints and not change it unless there is a compelling reason.
Compelling reasons might include the following:

* Your team may move from one-month sprints to two-week sprints in order to
  obtain more frequent feedback
* The annual holidays or end of the fiscal year make it more practical to run
  a longer sprint
* The product release occurs in one week, so a two-week sprint would be
  incongruous

The fact that the team cannot get all the work done within the current sprint
length is not a compelling reason to extend the sprint length. Neither is it
permissible to get to the last day of the sprint, realize you are not going to
be done, and lobby for an extra day or week. These are symptoms of dysfunction
and opportunities for improvement; they are not good reasons to change the
sprint length.

As a rule, therefore, if a team agrees to perform two-week sprints, all sprints
should be two weeks in duration. As a practical matter, most (but not all) teams
will define two weeks to mean ten calendar weekdays. If there is a one-day
holiday or training event during the sprint, it reduces the team’s capacity for
that sprint but doesn’t necessitate a sprint length change.

### The metronome effect

Sprints of the same duration provide the team with **cadence**, a regular,
predictable rhythm or heartbeat to a Scrum development effort. A steady, healthy
heartbeat allows the Scrum team and the organization to acquire an important
rhythmic familiarity with when things need to happen to achieve the fast,
flexible flow of business value. Having a regular cadence to sprints enables
people to “get into the zone,” “be on a roll,” or “get into a groove.”

Having a short sprint cadence also tends to level out the intensity of work.
Unlike a traditional sequential project where we see a steep increase in
intensity in the latter phases, each sprint has a similar intensity to that of
the other sprints.

Sprinting on a regular cadence also significantly reduces coordination overhead.
With fixed-length sprints everyone can predictably schedule the sprint-planning,
sprint review, and sprint retrospective activities for many sprints at the same
time. Because everyone knows when the activities will occur, the overhead
required to schedule them for a large batch of sprints is substantially reduced.

As an example, if you work on two-week sprints on a yearlong development effort,
you can send out the recurring event on everyone’s calendar for the next 26
sprint reviews. If you allowed sprint durations to vary from sprint to sprint,
imagine the extra effort you would need to coordinate the schedules of the
stakeholders on what might be just one or two weeks’ notice for an upcoming
sprint review! That assumes that you could even find a time that worked for the
core set of stakeholders, whose schedules are likely filled up many weeks ahead.

## Keeping your eye on the goal

An important Scrum rule states that once the sprint goal has been established
and sprint execution has begun, no change is permitted that can materially
affect the sprint goal.

Each sprint can be summarized by a sprint goal that describes the business
purpose and value of the sprint. Typically the sprint goal has a clear, single
focus, such as:

* Support customer service with new tooling
* Load and curate new geophysical data
* Integrate a new billing process into the application

There are times when a sprint goal might be multifaceted, for example, “Get
large-screen support working and support search by customer name.”

During sprint planning, the development team should help refine and agree to the
sprint goal and use it to determine the product backlog items that it can
complete by the end of the sprint. These product backlog items serve to further
elaborate the sprint goal.

### We commit to one another

The sprint goal is the foundation of a mutual commitment made by the team and
the product owner. The team commits to meeting the goal by the end of the
sprint, and the product owner commits to not altering the goal during the
sprint.

This mutual commitment demonstrates the importance of sprints in balancing the
needs of the business to be adaptive to change, while allowing the team to
concentrate and efficiently apply its talent to create value during a short,
fixed duration. By defining and adhering to a sprint goal, the Scrum team is
able to stay focused on a well-defined, valuable target.

### Clarifying the goal should not change the goal

Although the sprint goal should not be materially changed, it is permissible to
clarify the goal.

What constitutes a change? A change is any alteration in work or resources that
has the potential to generate meaningful wasted effort, egregiously disrupt the
flow of work, or substantially increase the scope of work within a sprint.
Adding or removing a product backlog item from a sprint or significantly
altering the scope of a product backlog item that is already in the sprint
typically constitutes change. The following example illustrates a change:

> Product owner: "Hey! When I said that we need to be able to search the
> inventory database by product name or description, I also meant that we should
> be able to search for it by a picture of the product!"

Adding the ability to search based on a picture likely represents substantially
more effort and almost certainly would affect the team’s ability to meet a
commitment to deliver search based on a name and description. In this case, the
product owner should consider creating a new product backlog item that captures
the search-by-picture feature. Then, they can add it to the product backlog to
be worked on in a later sprint.

What constitutes a clarification? Clarifications are additional details provided
during the sprint that assist the team in achieving its sprint goal. All of the
details associated with product backlog items might not be fully known or
specified at the start of the sprint. Therefore, it is completely reasonable for
the team to ask clarifying questions during a sprint and for the product owner
to answer those questions. The following example illustrates a clarification:

> Development team: "When you said that the matches for a product search should
>be displayed ina list, did you have a preference for how the list should be
>sorted?
>
>Product owner: "Yeah, by product name."
>
>Development team: "Great! We can do that!"

### The weight of change

It may appear that the no-goal-altering-change rule is in direct conflict with
the core Scrum principle that teams should embrace change. Though teams do
embrace change, they want to embrace it in a balanced, economically sensible
way.

The team invests in product backlog items to get them ready to be worked on in a
sprint. However, once a sprint starts, that investment in those product backlog
items has increased (because everyone spent time during sprint planning to
discuss and plan them at a task level). If someone wants to make a change after
sprint planning has occurred, they not only jeopardize the planning investment,
but the team also incur additional costs for having to replan any changes during
the sprint.

In addition, once we begin sprint execution, our investment in work increases
even more as product backlog items transition through the states of to do (work
not yet started), doing (work in process), and done (work completed).

Let’s say the product owner want to swap out feature X, currently part of the
sprint commitment, and substitute feature Y, which isn’t part of the existing
commitment. Even if no one has started working on feature X, the team still
incurs planning waste. In addition, feature X might also have dependencies with
other features in the sprint, so a change that affects feature X could affect
one or more other features, thus amplifying the negative effect on the sprint
goal.

Continuing that analysis, if work on feature X has already begun, in addition to
the wasted work completed, there can be other potential wastes. For example, all
of the work already performed on feature X might have to be thrown away. And the
team might have the additional waste of removing the partially completed work on
feature X, which may never get used in the future.

In addition to the direct economic consequences of waste, the economics can be
indirectly affected by the potential deterioration of team motivation and trust
that can accompany a change to a sprint. When the product owner makes a
commitment to not alter the goal and then violates the commitment, the team
naturally will be demotivated, which will almost certainly affect its desire to
work diligently to complete other prod- uct backlog items. In addition,
violating the commitment can harm the trust within the Scrum team, because the
development team will not trust that the product owner is willing to stick to
his commitments.

### Pragmatism in the face of perfection

The rule to not change a sprint goal is just that: a rule, not a law. The Scrum
team has to be pragmatic.

What if business conditions change in such a way that making a change to the
sprint goal seems warranted? Say a competitor launches its new product during a
sprint. After reviewing the new product, the product owner concludes that the
team needs to alter the goal it established for the current sprint because the
current tasks the team is doing is now far less valuable given what the
competitor has done. Should the team blindly follow the rule of no goal-altering
changes and not alter the sprint? Probably not.

What if a critical production system has failed miserably and some or all of the
people on the development team are the only ones who can fix it? Should the team
not interrupt the current sprint to fix it? Does the team tell the business that
they will fix the production failure first thing next sprint? Probably not.

In the end, being pragmatic trumps the "do not mess with the sprint" rule. The
team must act in a sensible way. Everyone on the Scrum team can appreciate that.
If the team changes the current sprint, they will experience the negative
economic consequences previously discussed. However, if the economic
consequences of the change are far less than the economic consequences of
deferring the change, making the change is the smart business decision. If the
economics of changing versus not changing are immaterial, no change to the
sprint goal should be made.

As for team motivation and trust, when a product owner has a frank, economically
focused discussion with the team about the necessity of the change, most teams
understand and appreciate the need, so the integrity of motivation and trust
is upheld.

### Stopping the sprint before the box is full

Should the sprint goal become completely invalid, the Scrum team may decide that
continuing with the current sprint makes no sense and advise the product owner
to abnormally terminate the sprint. When a sprint is abnormally terminated, the
current sprint comes to an abrupt end and the Scrum team gathers to perform a
sprint retrospective. The team then meets with the product owner to plan the
next sprint, with a different goal and a different set of product backlog items.

Sprint termination is used when an economically significant event has occurred,
such as a competitor’s actions that completely invalidate the sprint or product
funding being materially changed.

Although the product owner reserves the option to cancel each and every sprint,
it is rare that product owners invoke this option. Often there are less drastic
measures that a Scrum team can take to adjust to the situation at hand.
Remember, sprints are short, and, on average, the team will be about halfway
through a sprint when a change-causing situation arises. Because there may be
only a week or so of time left in the sprint when the change occurs, the
economics of terminating may be less favorable than just staying the course. And
many times it is possible to make a less dramatic change, such as dropping a
feature to allow time to fix a critical production failure instead of ending the
sprint.

It is important to realize that ending the sprint early, in addition to having a
negative effect on morale, is a serious disruption of the fast, flexible flow of
features and negates many of the benefits of consistent-duration sprints.
Terminating a sprint should be the last resort.

If a sprint is ended early, the Scrum team needs to decide the length of the
next sprint. Normally, there are three options. In a multi-team development
effort, choosing option 1 or option 2 is the preferred method.

1. Make the next sprint bigger than a normal sprint to cover the remaining time
   in the terminated sprint plus the time for the next full sprint.
1. Make the next sprint just long enough to get to the end date of the
   terminated sprint.
1. Stay with the original sprint length. This has the advantage of keeping a
   uniform sprint length throughout development (except for the terminated
   sprint, of course).

## What is "done"?

Conceptually the **definition of done** is a checklist of the types of work that
the team is expected to successfully complete before it can declare its work to
be ready for deployment for use.

Most of the time, a bare-minimum definition of done should yield a complete
slice of product functionality, one that has been designed, built, integrated,
tested, and documented and would deliver validated customer value. To have a
useful checklist, however, these larger-level work items need to be further
refined. For example, what does "tested" mean? Unit tested? Integration tested?
System tested? Platform tested? Internationalization tested? Are all of those
types of testing included in the definition of done?

Keep in mind that if you don’t do an important type of testing every sprint
(say, performance testing), you’ll have to do it sometime. Are you going to have
some specialized sprint in the future where the only thing you do is performance
testing? If so, and performance testing is essential to being“done,” you really
don’t have a usable product each sprint. And even worse, when you actually do
the performance testing at a later time and it doesn’t go quite as planned, not
only will you discover a critical problem very late in the process, but you will
also have to spend much more time and money to fix it at that time than if you
had done the performance testing earlier.

Sometimes the testing might take longer than the duration of a sprint. If this
occurs because the development team has accrued a huge manual testing debt, the
team needs to start automating its tests so that the testing can be completed
within a sprint like with the `mocha` library used in this course to write unit
tests for JavaScript applications.

Scrum teams need to have a robust definition of done, one that provides a high
level of confidence that what they build is of high quality and can be shipped.
Anything less robs the organization of the business opportunity of shipping at
its discretion and can lead to the accrual of **technical debt**.

### The evolution of done

You can think of the definition of done as defining the state of the work at the
end of the sprint. For many high-performance teams, the target end state of the
work enables it to be potentially shippable and that end state remains
relatively constant over the product's development.

Many teams, start out with a definition of done that doesn’t end in a state
where all features are completed to the extent that they could go live or be
shipped. For some, real impediments might prevent them from reaching this state
at the start of development, even though it is the ultimate goal. As a result,
they might start with a lesser end state and let their definition of done evolve
over time as organizational impediments are removed.

### Done and "done done"

Some teams have adopted the concept of “done” versus “done done.” Somehow done-
done is supposed to be more done than done! Teams that are unaccustomed to
really getting things done early and often are more likely to use "done done" as
a crutch. For them, using "done done" makes the point that being done (doing as
much work as they are prepared to do) is a different state from "done done"
(doing the work required for customers to believe it is done). Teams that have
internalized that you can be done only if you did all the work necessary to
satisfy customers don’t need to have two states; to them, done means "done
done"!

## What you've learned

In this reading, you learned a lot more about sprints. You now know that sprints

* are timeboxed;
* have a short and consistent duration
* have an unalterable goal; and,
* can only conclude using the team’s definition of done.

________________________________________________________________________________
# Requirements and User Stories

Scrum treats requirements very differently than traditional project planning.
With traditional product development, requirements are nonnegotiable, detailed
up front, and meant to stand alone. In Scrum, the details of a requirement are
negotiated through conversations that happen continuously during development and
are fleshed out just in time and just enough for the teams to start building
functionality to support that requirement.

With traditional product development, requirements are treated much as they are
in manufacturing: they are required, nonnegotiable specifications to which the
product must conform. These requirements are created up front and given to the
development group in the form of a highly detailed document. It is the job of
the development group, then, to produce a product that conforms to the detailed
requirements.

When a change from the original plan is deemed necessary, it is managed through
a formal change control process. Because conformance to specifications is the
goal, these deviations are undesirable and expensive. After all, much of the
work in process (WIP), in the form of highly detailed requirements (and all work
based on them), might need to be changed or discarded.

In contrast, Scrum views requirements as an important degree of freedom that we
can manipulate to meet our business goals. For example, if the product team is
running out of time or money, they can drop low-value requirements. If, during
development, new information indicates that the cost/benefit ratio of a
requirement has become significantly less favorable, that requirement can be
chosen to be dropped from the product. And if a new high-value requirement
emerges, the team has the ability to add it to the product, perhaps discarding a
lower-value requirement to make room.

The fact is, when developing innovative products, no one can create complete
requirements or designs up front by simply working longer and harder. Some
requirements and design will always emerge once product development is under
way; no amount of comprehensive up-front work will prevent that.

Thus, when using Scrum, the team doesn’t invest a great deal of time and money
in fleshing out the details of a requirement up front. Because the team expects
the specifics to change as time passes and as it learns more about what is being
built, the team avoids overinvesting in requirements that might later be
discarded. Instead of compiling a large inventory of detailed requirements up
front, the Scrum team create placeholders for the requirements, called product
backlog items (PBIs). Each product backlog item represents desirable business
value usually in the form of some desired functionality.

Initially the product backlog items are large (representing large swaths of
business value), and there is very little detail associated with them. Over
time, the team flows these product backlog items through a series of
conversations among the stakeholders, product owner, and development team,
refining them into a collection of smaller, more detailed PBIs. Eventually a
product backlog item is small and detailed enough to move into a sprint, where
it will be designed, built, and tested. Even during the sprint, however, more
details will be exposed in conversations between the product owner and the
development team.

While Scrum doesn’t specify any standard format for these product backlog items,
many teams represent PBIs as **user stories**. You don’t have to. Some teams
prefer "use case" format, and others choose to represent their PBIs in their own
custom formats.

## A PBI is a conversation waiting to happen

As a communication vehicle, requirements facilitate a shared understanding of
what needs to be built. They allow the people who understand what should be
created to clearly communicate their desires to the people who have to create
it.

Traditional product development relies heavily on written requirements, which
look impressive but can easily be misunderstood. A way to better ensure that the
desired features are being built is for the people who know what they want to
have timely conversations with the people who are designing, building, and
testing those features.

Scrum leverages conversation as a key tool for ensuring that requirements are
properly discussed and communicated. Verbal communication has the benefit of
being high-bandwidth and providing fast feedback, making it easier and cheaper
to gain a shared understanding. In addition, conversations enable bidirectional
communication that can spark ideas about problems and opportunities. Those are
discussions that would not likely arise from reading a document.

Conversation, however, is just a tool. It doesn’t replace all documents. In
Scrum, the product backlog is a “living document,” available at all times during
product development. Those who still want or must have a requirements
specification document can create one at any time, simply by collecting the
product backlog items and all of their associated details into a document
formatted however they like.

## Refining the refinements

With traditional product development all requirements must be at the same level
of detail at the same time. In particular, the approved requirements document
must specify each and every requirement so that the teams doing the design,
build, and test work can understand how to conform to the specifications. There
are no details left to be added.

Forcing all requirements to the same level of detail at the same time has many
disadvantages:

* They must predict all of the details early during product development when
  they have the least knowledge that they'll ever have about the prodcut and
  market.
* They treat all requirements the same regardless of their priority.
* They create a huge inventory of requirements that will likely be very
  expensive to rework or discard.
* They reduce the likelihood of using conversations to elaborate and clarify
  requirements because the requirements are "complete".

when using Scrum, not all requirements have to be at the same level of detail at
the same time. Requirements that the team will work on sooner will be smaller
and more detailed than ones that won’t be worked on for some time. Employing
this strategy of progressive refinement allows the entire team to work in a
just-in-time fashion.

## User stories

User stories are a convenient format for expressing the desired business value
for many types of product backlog items, especially features. User stories are
crafted in a way that makes them understandable to both business people and
technical people. They are structurally simple and provide a great placeholder
for a conversation. Additionally, they can be written at various levels of
granularity and are easy to progressively refine.

As well adapted to the needs of a development team as user stories might be,
don’t consider them to be the only way to represent product backlog items. They
are simply a lightweight approach that dovetails nicely with core agile
principles and our need for an efficient and effective placeholder.

So what exactly are user stories? Ron Jeffries offers a simple yet effective way
to think about user stories. He describes them as the three Cs: card,
conversation, and confirmation.

### The card

The card idea is pretty simple. People originally wrote (and many still do) user
stories directly on 3 × 5-inch index cards or sticky notes.

A common template format for writing user stories uses the "Conextra format"
known as the "as a-I want-so that" format. That format specifies a class of
users (the user role), what that class of users wants to achieve (the goal), and
why the users want to achieve the goal (the benefit). The “so that” part of a
user story is optional, but unless the purpose of the story is completely
obvious to everyone, it should be included with every user story.

> **As a** typical member, **I want** to see unbiased reviews of a restaurant
> near an address **so that** I can decide where to go for dinner.

The card isn’t intended to capture all of the information that makes up the
requirement. In fact, most agile methodologies deliberately use small cards with
limited space to promote brevity. A card should hold a few sentences that
capture the essence or intent of a requirement. It serves as the placeholder for
more detailed discussions that will take place among the stakeholders, product
owner, and development team.

### The conversation

The details of a requirement are exposed and communicated in a conversation
among the development team, product owner, and stakeholders. The user story is
simply a promise to have that conversation.

The conversation is typically not a one-time event, but rather an ongoing
dialogue. There can be an initial conversation when the user story is written,
another conversation when it’s refined, yet another when it’s estimated, another
during sprint planning (when the team is diving into the task-level details),
and finally, ongoing conversations while the user story is being designed,
built, and tested during the sprint.

One of the benefits of user stories is that they shift some of the focus away
from writing and onto conversations. These conversations enable a richer form of
exchanging information and collaborating to ensure that the correct requirements
are expressed and understood by everyone.

Although conversations are largely verbal, they can be and frequently are
supplemented with documents. Conversations may lead to a UI sketch, or an
elaboration of business rules that gets written down. Scrum does not do away
with all other documentation in favor of user stories. User stories are simply
a good starting point for eliciting the essence of the requirement.

### The confirmation

A user story also contains confirmation information in the form of conditions of
satisfaction. These are acceptance criteria that clarify the desired behavior.
They are used by the development team to better understand what to build and
test and by the product owner to confirm that the user story has been
implemented to his satisfaction.

If the team uses physical index cards, the front of the card has a few-line
description of the story, the back of the card could specify the conditions of
satisfaction.

These conditions of satisfaction can be expressed as high-level acceptance
tests. However, these tests would not be the only tests that are run when the
story is being developed. In fact, for the handful of acceptance tests that are
associated with a user story, the team will have many more tests (perhaps 10 to
100 times more) at a detailed technical level that the product owner doesn’t
even know about.

The acceptance tests associated with the story exist for several reasons. First,
they are an important way to capture and communicate, from the product owner’s
perspective, how to determine if the story has been implemented correctly.

These tests can also be a helpful way to create initial stories and refine them
as more details become known.

## How much detail is enough?

User stories are an excellent vehicle for carrying items of customer or user
value through the Scrum value-creation flow. However, if your team has only one
story size (the size that would comfortably fit within a short-duration sprint),
it will be difficult to do higher-level planning and to reap the benefits of
progressive refinement.

Small stories used at the sprint level are too small and too numerous to support
higher-level product and release planning. At these levels you need fewer, less
detailed, more abstract items. Otherwise, you’ll be mired in a swamp of mostly
irrelevant detail. Imagine having 500 very small stories and being asked to
provide an executive-level description of the proposed product to secure your
funding. Or try to prioritize among those 500 really small items to define the
next release.

Also, if there is only one (small) size of story, we will be obligated to define
all requirements at a very fine-grained level of detail long before we should.
Having only small stories precludes the benefit of progressively refining
requirements on a just- enough, just-in-time basis.

Fortunately, user stories can be written to capture customer and user needs at
various levels of abstraction. The largest would be stories that are a few to
many months in size and might span an entire release or multiple releases. Many
people refer to these as **epics**, alluding to the idea that they are Lord of
the Rings or War and Peace size stories. Epics are helpful because they give a
very big-picture, high-level overview of what is desired.

> **Epic**: As someone that regularly uses the restaurant review product, I want
> to train the system on what types of cuisine I prefer so that it will know
> what restaurants to use when filtering reviews on my behalf.

You would never move an epic into a sprint for development because it is way too
big and not very detailed. Instead, epics are excellent placeholders for a large
collection of more detailed stories to be created at an appropriate future time.

The next-size stories are those that are often on the order of weeks in size and
therefore too big for a single sprint. Some teams might call these **features**.

The smallest forms of user stories are those typically refered to as
**stories**.

## Good stories are INVESTments

Good user stories exhibit six aspects. They are

* Independent of one another
* Negotiable
* Valuable to the people using the product
* Estimable by the development team
* Small enough to fit in a sprint
* Testable to know when its done

### Celebrate your independence

As much as is practical, user stories should be independent or at least only
loosely coupled with one another. Stories that exhibit a high degree of
interdependence com- plicate estimating, prioritizing, and planning. When
applying the independent criteria, the goal is not to eliminate all
dependencies, but instead to write stories in a way that minimizes dependencies.

### Negotiate for profit

The details of stories should also be negotiable. Stories are not a written
contract in the form of an up-front requirements document. Instead, stories are
placeholders for the conversations where the details will be negotiated.

Good stories clearly capture the essence of what business functionality is
desired and why it is desired. However, they leave room for the product owner,
the stakeholders, and the team to negotiate the details.

This negotiability helps everyone involved avoid the us-versus-them, finger-
pointing mentality that is commonplace with detailed up-front requirements docu-
ments. When stories are negotiable, developers can’t really say, “Hey, if you
wanted it, you should have put it in the document,” because the details are
going to be negoti- ated with the developers. And the business people can’t
really say, “Hey, you obviously didn’t understand the requirements document
because you built the wrong thing,” because the business people will be in
frequent dialogue with the developers to make sure there is shared clarity.
Writing negotiable stories avoids the problems associated with up-front detailed
requirements by making it clear that a dialogue is necessary.

A common example of where negotiability is violated is when the product owner
tells the team how to implement a story. Stories should be about what and why,
not how. When the how becomes nonnegotiable, opportunities for the team to be
innovative are diminished.

There are times, however, when how something is built is actually important to
the product owner. For example, there might be a regulatory obligation to
develop a feature in a particular way, or there might be a business constraint
directing the use of a specific technology. In such cases the stories will be a
bit less negotiable because some aspect of the “how” is required. That’s OK; not
all stories are fully negotiable, but most stories should be.

### Valuable for everyone

Stories need to be valuable to a customer, user, or both. Customers (or
choosers) select and pay for the product. Users actually use the product. If a
story isn’t valuable to either, it doesn’t belong in the product backlog.

How about stories that are valuable to the developers but aren’t of obvious
value to the customers or users? Is it OK to have **technical stories**.

> **Technical story**: As a developer, I want to migrate the system to work on
> the latest version of ReactJS so that we are not operating on a stale version
> of the UI library and get stuck in an expensive upgrade later.

In the case of the “Migrate to New Version of ReactJS" story, the product owner
might not initially understand why it is valuable to change databases. However,
once the team explains the risks of continuing to develop on an unsupported
version of a database, the product owner might decide that migrating databases
is valuable enough to defer building some new features until the migration is
done. By understanding the value, the product owner can treat the technical
story like any other business-valuable story and make informed trade-offs. As a
result, this technical story might be included in the product backlog.

In practice, though, most technical stories should not be included in the
product backlog. Instead, these types of stories should be tasks associated with
getting normal stories done. If the development team has a strong definition of
done, there should be no need to write stories like these, because the work is
implied by the definition of being done.

### Estimations are the lifeblood of prediction

Stories should be estimable by the team that will design, build, and test them.
Estimates provide an indication of the size and therefore the effort and cost of
the stories (bigger stories require more effort and therefore cost more money to
develop than smaller stories).

Knowing a story’s size provides actionable information to the Scrum team. The
product owner, for example, needs to know the cost of a story to determine its
final priority in the product backlog. The Scrum team, on the other hand, can
determine from the size of the story whether additional refinement or
disaggregation is required. A large story that we plan to work on soon will need
to be broken into a set of smaller stories.

If the team isn’t able to size a story, the story is either just too big or
ambiguous to be sized, or the team doesn’t have enough knowledge to estimate a
size. If it’s too big, the team will need to work with the product owner to
break it into more manageable stories. If the team lacks knowledge, some form of
exploratory activity will be needed to acquire the information.

### Small enough to accomplish

Stories should be sized appropriately for when you plan to work on them. Stories
worked on in sprints should be small. If you’re doing a several-week sprint, you
will want to work on several stories that are each a few days in size. If you
have a two-week sprint, you don’t want a two-week-size story, because the risk
of not finishing the story is just too great.

So ultimately you need small stories, but just because a story is large, that
doesn’t mean it’s bad. Let’s say you have an epic-size story that you aren’t
planning to work on for another year. Arguably that story is sized appropriately
for when you plan to work on it. In fact, if you spent time today breaking that
epic down into a collection of smaller stories, it could easily be a complete
waste of your time. Of course, if you have an epic that you want to work on in
the next sprint, it’s not sized appropriately and you have more work to do to
bring it down to size. You must consider when the story will be worked on when
applying this criterion.

### Test makes the world go 'round

Stories should be testable in a binary way: they either pass or fail their
associated tests. Being testable means having good acceptance criteria (related
to the conditions of satisfaction) associated with the story, which is the
“confirmation” aspect of a user story.

It may not always be necessary or possible to test a story. For example,
epic-size stories probably don’t have tests associated with them, nor do they
need them because you don’t directly build the epics.

Also, on occasion there might be a story that the product owner deems valuable,
yet there might not be a practical way to test it. These are more likely to be
so-called **nonfunctional requirements**, such as “As a user, I want the system
to have 99.999% uptime.” Although the acceptance criteria might be clear, there
may be no set of tests that can be run when the system is put into production
that can prove that this level of uptime has been met, but the requirement is
still valuable as it will drive the design.

## What you've learned

A hard thing is having proper requirements as a team to guide your work. In
this reading, you learned about user stories and how they should be INVESTable.
Some of this may not make sense, just now, and that's ok. It will make sense as
you run into more real-world requirements in this course, as well as when you
move into real-world development scenarios.

________________________________________________________________________________
# Censor Project

Time to practice some Node Input/Output! In this project, you'll utilize both
user I/O and file I/O to create a program that censors forbidden words that are
specified in a dictionary file.

The solution for this project is available at the end of these instructions. Be
sure to give it an honest shot before you take a peek!

## The Objective

Let's begin by taking a bird's-eye view of how the final product should behave.
Let's say that we had a `forbidden-dictionary.txt` containing the following
words:

```plaintext
potato
tomato
cat
strange
real
park
```

When we execute the program, `censor.js`, the user will be prompted to enter a
sentence. The program will respond with the same sentence, but with the
forbidden words censored. A censored word will have its vowels replaced with
stars (\*). Below is an example of the program at runtime. For clarity, we have
wrapped the user's input in -dashes-:

```plaintext
$ node censor.js
Enter a sentence to be censored: -what a really strange place to park a car-
what a really str*ng* place to p*rk a car
```

## Phase 1: Setting Up and Censoring Sentences

We'll be designing this project from scratch, so begin by creating directory
named `censor-project` and `cd` into it. This will be our working directory for
the duration of the project. Inside of your working directory, create two files:
`censor.js` and `forbidden-dictionary.txt`. Your directory should have the
following structure:

```plaintext
/censor-project
├── censor.js
└── forbidden-dictionary.txt
```

### Creating a dictionary

Inside of your `forbidden-dictionary.txt` use VSCode to write some words that
we'll eventually censor. For the rest of these instructions, you can assume that
our `forbidden-dictionary.txt` contains exactly:

```plaintext
potato
tomato
cat
strange
real
park
```

Note that we have typed each forbidden word on it's own line, be sure to do the
same! Until phase 2, we'll put this dictionary file to the side.

 Our program should censor a word by replacing its vowels with stars
(\*). To keep things nice and tidy, create a function `starVowels` that accepts
a word as an argument and returns that word with all of its vowels replaced with
stars. We leave the implementation detail up to you. Your function should be
able to satisfy the following examples:

```javascript
console.log(starVowels("caterpillar")); // 'c*t*rp*ll*r'
console.log(starVowels("snowstorm")); // 'sn*wst*rm'
console.log(starVowels("programmer")); // 'pr*gr*mm*r'
```

Test your function thoroughly with the above test cases and feel free to test it
with other arguments.

You may have noticed that we are assuming the `starVowels` function will be
passed a single word _and not_ a sentence containing many words. This is because
our final product should be deliberate in _which_ words will be targeted.
`starVowels` is just a helper function that will be used within our main
`censorSentence` function. Create a function named `censorSentence` that accepts
two arguments: a sentence and an array of strings. The function should return a
new sentence where words of the original sentence are censored if they are found
inside of the array argument. You should use `starVowels` as a helper function,
but we leave the rest of the implementation detail to you. You can assume that
won't contain any punctuation. Your `censorSentence` should satisfy the
following behavior:

```javascript
console.log(censorSentence("what a wonderful life", ["wonderful", "tree"]));
// 'what a w*nd*rf*l life'

console.log(
  censorSentence("hey programmer why the long face", [
    "long",
    "programmer",
    "hey"
  ])
);
//  'h*y pr*gr*mm*r why the l*ng face'
```

Great! We're done with the censoring logic of our program. Notice how we
designed a _function_ to perform the act of censoring. This keeps our code
modular. We didn't even have to deal with any I/O yet! In general, you'll want
to keep your code loosely coupled so that it is easy to test and maintain. Now
that we have `censorSentence`, for the rest of the project we will concern
ourselves with passing the arguments dynamically into `censorSentence`. That is,
we have two open questions:

- how can we get the sentence argument for `censorSentence` from the user?
- how can we get the array argument for `censorSentence` from the dictionary
  file?

Before moving onto the next phase, don't forget to delete any test calls to
`censorSentence`.

## Phase 2: Interacting with the user

Let's tackle prompting the user for a sentence using the `readline` module.
You've seen this pattern a few times now. Reference the [readline
docs][readline-docs] to do three things:

1. import the readline module
2. create an interface so that the terminal can be used for I/O
3. use the `question` method to ask the user to enter a sentence; once they hit
   enter, repeat the sentence to them and `close` the interface

Here is an example of how the program should behave if you have satisfied the
three points above. For clarity, we've denoted the user's input with -dashes-:

```plaintext
$ node censor.js
Enter a sentence to be censored: -hey programmers-
hey programmers
```

Once you are satisfied with the user input, you can even begin to mock out how
the user's sentence will be modified. Instead of simply repeating their sentence
once they hit enter, use your `censorSentence` function to censor some arbitrary
words of their sentence. This means you'll have to pass a hard-coded array as
the second argument to `censorSentence`.

Just one more piece of the puzzle! Instead of using the static dictionary array
that we just hard-coded, we'll want to refactor our code to get this data from a
separate file.

## Phase 3: Parsing the dictionary

This part is tricky, so read the full instructions for this entire phase before
writing any more code.

Our task now is to use the `fs` module to read the contents of the
`forbidden-dictionary.txt` file that we created initially. To do this, feel free
to reference the [docs for readFile][fs-read-file-docs] and our previous reading
on file I/O. Like usual, you should use 'utf8' as the encoding and print any
potential errors that may occur when the file is read.

Since both the readline `question` method and fs `readFile` method are
asynchronous, you'll need to utilize _callback chaining_ in order to read the
`forbidden-dictionary.txt` file _after_ the user has entered their sentence.
Recall that _callback chaining_ is implemented by nesting callbacks. This means
that you'll have to call `readFile` from _within_ the callback that is passed
into `question`.

Remember that our `censorSentence` needs to be passed an array of strings to be
censored as its second argument. However, if we read the contents of the
`forbidden-dictionary.txt`, we will receive one long string that contains all of
the words. You'll need to reconcile this difference. How did we overcome this in
the previous reading? We'll leave it to you!

Test your final product by entering different sentences at runtime and changing
what words are included in the dictionary file. Amazing!

**Ask an instructor for a code review.**

## Bonus: Pick a file, any file

We recommend completing all other core projects in this lesson before going back
and working on this bonus or any other bonus features.

Create a copy of your entire `censor-project` directory so that you have a
working version of the core project to reference. You may find yourself making
some drastic changes in this bonus (but hopefully not :)).

In the core phases of this project, we hard-coded the location of the dictionary
file to be `forbidden-dictionary.txt`. Your objective for this bonus is to now
allow the user to specify which dictionary file they would like to use after
they enter their sentence. For example, let's say that we added some additional
files to our working directory so that it had the following structure:

```plaintext
/censor-project
├── censor.js
├── forbidden-dictionary.txt
├── bonus-dictionary-a.txt
└── some_deeper_directory
    └── bonus-dictionary-b.txt
```

We recommend that you mimic the structure above. Feel free to populate the
`.txt` files with whatever words you like (be sure to separate each word with a
newline).

 In this
project, you will be creating a program that will edit a file, replacing all
occurrences of a given string with another string. You may be familiar with this
"global replace" feature as it is frequently supported by word processors.
VSCode also supports this functionality through the `cmd/ctrl + f` shortcut.
Your goal is to implement this feature for yourself using JavaScript and Node's
`fs` module!

The solution for this project is available at the end of these instructions. Be
sure to give it an honest shot before you take a peek!

## The Objective

Let's take a bird's-eye-view of how our final product will behave. Let's say
that we had an `essay.txt` file that had the following random text:

```plaintext
Lorem ipsum dolor amet single-origin coffee trust fund organic
normcore, wayfarers narwhal fam hashtag ugh VHS af. Try-hard
brooklyn you probably haven't heard of them stumptown. Coloring book
selfies pickled plaid small batch butcher beard fixie disrupt
schlitz irony. Offal deep v meditation squid.

Truffaut ramps VHS, pabst air plant la croix godard authentic
everyday carry street art deep v shaman. 3 wolf moon cloud bread
brooklyn health goth meditation literally salvia, tumblr chambray.
Taiyaki slow-carb distillery, seitan food truck drinking vinegar
hexagon gastropub offal gluten-free banjo.
```

For the remainder of these instructions, you may assume that our **original**
`essay.txt` contains the text above, before the program edits it.

When we run our program, `global-replace.js`, we should be able to specify
command line arguments for the file to edit, the target string, and the
replacement string. For example, we should be able to use our program to edit
`essay.txt` by replacing every occurrence of `'oo'` with `'HIYAAAH'` by
executing the following command in our terminal:

```plaintext
$ node global-replace.js essay.txt oo HIYAAAH
```

Afterwards, the **modified** contents of `essay.txt` will be:

```plaintext
Lorem ipsum dolor amet single-origin coffee trust fund organic
normcore, wayfarers narwhal fam hashtag ugh VHS af. Try-hard
brHIYAAAHklyn you probably haven't heard of them stumptown. Coloring bHIYAAAHk
selfies pickled plaid small batch butcher beard fixie disrupt
schlitz irony. Offal deep v meditation squid.

Truffaut ramps VHS, pabst air plant la croix godard authentic
everyday carry street art deep v shaman. 3 wolf mHIYAAAHn cloud bread
brHIYAAAHklyn health goth meditation literally salvia, tumblr chambray.
Taiyaki slow-carb distillery, seitan fHIYAAAHd truck drinking vinegar
hexagon gastropub offal gluten-free banjo.
```

With this high-level goal in mind, let's jump in!

> **Did you know?** The random text we used above is a variation of [lorem
> ipsum][lorem-ipsum-wiki]. Lorem ipsum is meaningless placeholder text that is
> commonly used to temporarily fill space. You'll likely be seeing lorem ipsum
> text on websites that are in development when actual data isn't available yet.
> To generate our essay.txt, we used lorem hipsum.

## Phase 1: Setup and Command Line Arguments

We'll create this project from scratch. Begin by creating a
`global-replace-project` directory. This will be our working directory for the
entirety of this project, so `cd` into it. Inside of your directory, create two
files: `global-replace.js` and `essay.txt`. We recommend that you use VSCode to
populate the `essay.txt` file with our original text from above so you can
follow our examples. Your directory structure should be:

```plaintext
/global-replace-project
├── essay.txt
└── global-replace.js
```

We'll put `essay.txt` aside until the second phase.

### Taking command line arguments

Let's work on reading command line arguments when the user executes the program.
You may have noticed that the final product output provided in the introduction
shows the user specifying arguments along with the command that we classically
use to run the program, `node global-replace.js`. Previously, we've been
collecting input _during runtime_ with the `readline` module, but that is not
the case here.

Taking command line arguments is very simple in Node and doesn't require any
modules. To do this, we'll reference the `process.argv` value that is available
when a `.js` file is executed with Node. See this for yourself by writing this
simple line at the top of your `global-replace.js` file:

```javascript
console.log(process.argv);
```

If you execute your script with `node global-replace.js`, you should see an
array containing the paths to the Node runtime and your script. Here is what the
output looked like on our machine:

```plaintext
[ '/usr/local/bin/node',
  '/Users/az/Desktop/global-replace-project/global-replace.js' ]
```

See where this is heading? If you execute your script with additional arguments
separated by spaces, such as `node global-replace.js potato.txt tomato squash`,
the `process.argv` array will contain those additional arguments:

```plaintext
[ '/usr/local/bin/node',
  '/Users/az/app_academy/Module-JavaScript/projects/node/global-replace-project/global-replace-project/global-replace.js',
  'potato.txt',
  'tomato',
  'sqash' ]
```

We'll assume that the user will specify the `TARGET_FILE`, `OLD_STR`, and
`NEW_STR` arguments in that order. In your script, assign three global `const`
variables to contain these arguments from `process.argv`. Bragging rights
awarded if you do this with _array destructuring_.

> **Did you know?** It is convention in many programming languages to style
> global constant variables with CONSTANT_CASE.

The global `process` object is specific to Node and is not available in the
browser runtime. We use it here for [process.argv][process-argv] and we used it
previously for `process.stdin` and `process.stdout` when creating interfaces for
the readline module in past projects.

One more note before moving on. Depending on how you decide to implement things
during this project, your program may require you to enter all three command
line arguments in order to execute. That is, you'll have to execute your code
with `node global-replace.js arg1 arg2 arg3` and not just
`node global-replace.js`.

## Phase 2: Simple string replacement and file reading

For now, we won't be utilizing the command line arguments. Let's work on
implementing logic to perform global replacement on a string.

In `global-replace.js`, write a function named `replace` that accepts three
string arguments. The function should return a modified version of the first
string where all occurrences of the second string are replaced with the third.
Your function should satisfy the following examples:

```javascript
console.log(replace("what a great program", "a", "o")); // 'whot o greot progrom'
console.log(replace("what a great program", "gr", "d")); // 'what a deat prodam'
console.log(replace("have a nice day", "a nice", "an amazing")); // 'have an amazing day'
```

Now that the `replace` function is complete, delete all test calls you made to
it. We'll set this function aside for a moment to focus our attention on
performing some file I/O on the `TARGET_FILE`.

Import the `fs` module and use the `readFile` method to read the `TARGET_FILE`.
Look up the documentation if you forgot how to do this. Use 'utf8' as the
encoding and print any errors that occur during the read. Also simply print out
the contents of the file to check if everything is working as it should.

Upon running `node global-replace.js essay.txt`, you should see the contents of
your file printed out. If you do, take things a step further by printing out a
modified version of the file contents where all instances of `OLD_STR` are
replaced with `NEW_STR`. Recall that the data from `readFile` will be read as
one long string since we specified 'utf8'; this is a perfect fit for our
`replace` function. Do not worry about overwriting the contents of the file yet,
simply `console.log` the modified contents.

Test your work by passing three valid command line arguments. For example here
is what our program printed upon running `node global-replace.js essay.txt a X`:

```plaintext
Lorem ipsum dolor Xmet single-origin coffee trust fund orgXnic
normcore, wXyfXrers nXrwhXl fXm hXshtXg ugh VHS Xf. Try-hXrd
brooklyn you probXbly hXven't heXrd of them stumptown. Coloring book
selfies pickled plXid smXll bXtch butcher beXrd fixie disrupt
schlitz irony. OffXl deep v meditXtion squid.

TruffXut rXmps VHS, pXbst Xir plXnt lX croix godXrd Xuthentic
everydXy cXrry street Xrt deep v shXmXn. 3 wolf moon cloud breXd
brooklyn heXlth goth meditXtion literXlly sXlviX, tumblr chXmbrXy.
TXiyXki slow-cXrb distillery, seitXn food truck drinking vinegXr
hexXgon gXstropub offXl gluten-free bXnjo.
```

Almost done! Now we just need to take this new string and rewrite the
`TARGET_FILE`.

## Phase 3:

Let's take a quick recap of what we are dealing with before we put the finishing
touch on this project. So far, our program _asynchronously_ reads data from the
`TARGET_FILE` and we use `replace` to modify the incoming data from the
`TARGET_FILE`. In order to write this modified data back into the `TARGET_FILE`,
we need to utilize _callback chaining_ because all of these file operations are
asynchronous!

Utilize callback chaining and [writeFile][write-file] to rewrite the
`TARGET_FILE` with the new data. This means you will have to call `writeFile`
within the callback for `readFile`. Be sure to use 'utf8' and print any errors
that occur during the write.

Test your final product by running your program a few times, replacing different
substrings. Verify that the contents of `essay.txt` are changed. Test things
further by creating your own `.txt` files to edit! Remember that we can specify
any `TARGET_FILE` through command line arguments, so your program is very
dynamic!

**Ask an instructor for a code review.**

## Bonus: Replacing the first 'n' occurrences

We recommend completing all other core projects in this lesson before going back
and working on this bonus or any other bonus features.

For this bonus, we'll give our program the ability to accept an additional
fourth argument representing the number of occurrences to replace. In other
words, we should be able to run the program with the following arguments:

```plaintext
node global-replace.js essay.txt i hello 3
```

This should edit the `essay.txt` file, replacing the first 3 occurrences of 'i'
with 'hello'. This new argument should be optional, so if we run the program
with:

```plaintext
node global-replace.js essay.txt i hello
```

It should replace **all** occurrences of 'i' with 'hello', as it did in the core
project.

We'll leave the implementation up to you! Good luck.

[lorem-ipsum-wiki]: https://en.wikipedia.org/wiki/Lorem_ipsum
[process-argv]:
  https://nodejs.org/docs/latest/api/process.html#process_process_argv
[write-file]:
  https://nodejs.org/api/fs.html#fs_fs_writefile_file_data_options_callback

________________________________________________________________________________
# Repo Madness Project: Managing Your Code With Git

Now that you've developed an understanding of what Git is and why we use it,
let's get to work! We're going on a time-traveling adventure with Git as our
vehicle.

We'll review:
- Creating and navigating a Git repository,
- Adding and changing our files,
- Browsing a timeline of our changes,
- and rewriting history!

## Phase 1: Laying the foundation

To get started, we need to create a timeline. For this project, we'll be working
with plain old JavaScript objects. As we move forward and back in time, we'll
observe changes in our objects.

Let's start by creating our own timeline from scratch. Once we've practiced
this, we'll import an already-organized timeline from GitHub and use it as a
base for our adventure.

### Phase 1a: Create your own reality

We'll take control of time and space by creating our own timeline first! Create
a new Git repository called "my-custom-timeline". Within this repository, create
a file called `me.js`. Add an empty JavaScript object to your `js` file.

Since this is your timeline, let's add some important dates in your own life.
We'll start (as we all do!) with your birthday. Inside `me.js`, add a couple
properties to your JavaScript object. Add an `"age"` property and set it to `0`,
and add at least one more property to describe yourself: maybe `"hairColor":
"brown"`. Add these changes to your repo and commit them with a message like
"Add birthday (1987)". Including the year in your commit messages for this phase
will make it easy to follow your timeline in your commit history later on.

Time marches on - and so should you! Think of at least three more significant
dates in your life. These could be graduations, vacations, or even the day your
cohort started at ! With each event, make appropriate changes to
`me.js`. Update your `"age"` value and modify the object to reflect what changes
you experienced. For example: if one of your significant dates was a wedding,
you might add a `"spouse"` property.

Your repository should end up with at least four commits: one for "Initial
commit (before I came along)" and three for events on your timeline. Remember:
we're focusing on Git, so you should use clear commit messages in imperative
voice that make it easy to look back over your changes.

Once you're satisfied, take a look at your Git commit log. It should read like a
(very) abridged biography. This is what every repo's commit history should look
like: a concise history of significant changes as the project has evolved.

### Phase 1b: A shared experience

Let's move on to more advanced techniques. For this, we'll need to ensure we're
all starting with the same timeline. Leave your `my-custom-timeline` directory
and download the skeleton for this project below. After opening the .ZIP file,
you should end up with a new directory containing a single file: `earth.js`.
Navigate into this directory using `cd my-shared-timeline` from your terminal.
We'll build on this shared timeline from here on.

Take a look at `earth.js`. You'll see a JavaScript object containing the name of
our planet as well as **very** rough population estimates. This object will
include all the data we track on our new timeline.

> A quick aside: Our "population estimates" are pretty silly, but you should be
> focused on the "time-traveling" techniques of Git and not the historical
> authenticity of these examples. If you'd like to know more, we've loosely
> based our original timeline on the ["Timeline of the evolutionary history of
> life"][Life timeline source]. Feel free to draw from this resource for ideas
> as you customize your own history!

## Phase 2: Exploring our timeline

What we're seeing in `earth.js` is representative of the present.  Notice how our log here reads like a very simplified
history of Earth. Let's travel back and see what we can find!

## Phase 2a: Looking around without moving

Before we set off, let's look back over our timeline so we know what we're
getting into. Using `git diff`, compare your current commit to the first commit
in the repository's history. You should see a single line change: one
subtraction, one addition, both on the same line.

Of course, we can tell from the commit history that more has changed than just
that one line! `git diff` is a great tool for seeing the **overall** change
since a particular point in time, but to see **all** the changes since that
time, we have to look at each commit separately.

Try comparing more commits in the repo. Between which diffs can you see the most
changes? Are there any commits with a _net neutral_ (no change) diff between
them?

Notice that nothing changes about your current state as you view diffs. Diffing
is a safe way of browsing a repo without changing your position, so it's great
as a "quick reference" if you need to check a detail in your repo's history.

## Phase 2b: Our first steps through time

Now, let's relocate and see each commit with its respective context. Using `git
checkout`, navigate to the first commit in the repository. How many of each
population are there at this time?

Let's jump back to the present! Check out the `master` ref to return to the
latest commit on this branch.

Try stepping through your history one commit at a time and observing how
`earth.js` changes. As you browse through your repo, keep an eye on your `git
log` output as well. Notice that you're unable to see commits ahead of your
current position - only those that were created in the past.

Remember: checking out a commit is non-destructive, so it's a safe way to
explore the state of a repository at a given point in time.

## Phase 3: Changing history

Checking out and comparing commits are both safe, non-destructive ways of
browsing your repo. What if we would rather do some damage and make real
changes? It's time to get messy!

Before we make any changes, we need to ensure we're not damaging our original
timeline. The safest way to make isolated changes to a repo is via a branch.
Let's create a new branch titled
`alternate-timeline-for-<your-github-username-here>`. Use `git status` to make
sure you've checked your new branch out before continuing.

Let's go back a few million years and make a change that will have serious
effects on our timeline. Using `git reset`, travel back to 4.5 million years
ago. Let's assume mankind had never come along to drive the woolly mammoth to
extinction. How many mammoths might there be today: 200+? 500+? Even more?

Update `earth.js` to include your estimate of how many mammoths might roam the
Earth today. Add this change to your staging area and commit it with a message
like "Today: Mammoths rule the world".

 You should see that the previous two
commits for "2.5 million years ago" and "Today" have been totally replaced by
your one new commit for "Today". Congratulations: you've changed history! You
can use `git diff` to compare your new branch with the `master` branch.

 You can add additional
commits and rewrite different parts of history as you see fit. Did the dinosaurs
live into modern times, letting you own a cool Tyrannosaurus Rex pet? It's up to
you!

When you're done, let's share your new vision with the world. We'll do this with
the help of GitHub.

First, we need to make sure your alternative timeline is compatible with our
original timeline. Merge the `master` branch into your branch and resolve any
conflicts. Since we're only changing numbers, this should be an easy conflict to
resolve: just remove the "old" numbers from the `master` branch, replace them
with your "new" population totals, and make sure you don't leave any merge
conflict artifacts (`<<<<<`, `=====`, or `>>>>>`) hanging around in there! `git
status` will provide some helpful details if you're unsure how to proceed.

Once you've successfully merged `master` into your branch, we need to
go to GitHub and create a new remote for our project.

Go to GitHub and create a new repo under your account
called `my-shared-timeline`. This will serve as our new "remote". You can
give it a description, but do not create a README file. The repository can be
public or private.

Once you have created this repo, you need to tell your local git repository
to use this Github repo as its remote. Do the following on your local terminal:
`git checkout master`
`git remote add origin https://github.com/<USERNAME>/my-shared-timeline.git`
`git push -u origin master`

A note on HTTPS vs. SSH: For now, you should use HTTPS for cloning and pushing
GitHub repos. This will require you to enter your username and password.
Eventually, we will have you generate SSH keys which allow you to store a
unique key on your local machine that GitHub reads for authentication purposes.

Now that we have our remote repository, we need to push up our new branch and
open a pull request. You'll want `master` to be the _base_ branch and
your branch to be the _compare_ branch. Don't forget to add a short description
of the changes you made. You can review your changes on GitHub before opening
the PR.

## What We've Learned

Whew! We're safely back in the present. Git is a powerful tool that lets us
explore and manage the history of our repository with ease. While playing with
dinosaurs can be fun, you've also been flexing your programming muscles by:

- Creating a new code repository from scratch,
- Tracking changes to a JavaScript file over time,
- Practicing how to explore an unfamiliar repo,
- Branching off a shared repo to make your own changes,
- and sharing those changes with your team and/or the world via GitHub.

These are skills you'll use on the job many times per day, so we'll continue to
get lots of practice together as you incorporate Git in upcoming projects.

[Life timeline source]: https://en.wikipedia.org/wiki/Timeline_of_the_evolutionary_history_of_life

________________________________________________________________________________
# WEEK-03 DAY-3<br>*Unix* {ignore=true}
________________________________________________________________________________
# Command Line Interface Basics Lesson Learning Objectives

Below is a complete list of the terminal learning objectives for this lesson.
When you complete this lesson, you should be able to perform each of the
following objectives. These objectives capture how you may be evaluated on the
assessment for this lesson.

1. Given a folder structure diagram, a list of 'cd (path)' commands and target
files, match the paths to the target files.
2. Create, rename, and move folders using unix command line tools.
3. Use grep and | to count matches of a pattern in a sample text file and save
result to another file.
4. Find what -c, -r, and -b flags do in grep by reading the manual.
5. Identify the difference in two different files using diff.
6. Open and close nano with and without saving a file.
7. Use ‘curl’ to download a file.
8. Read the variables of $PATH.
9. Explain the difference between .bash_profile and .bashrc.
10. Create a new alias by editing the .bash_profile.
11. Given a list of common scenarios, identify when it is appropriate and safe
to use sudo, and when it is a dangerous mistake.
12. Write a shell script that greets a user by their $USER name using echo.
13. Use chmod to make a shell script executable.list of the terminal learning objectives for this lesson.
When you complete this lesson, you should be able to perform each of the
following objectives. These objectives capture how you may be evaluated on the
assessment for this lesson.

1. Given a folder structure diagram, a list of 'cd (path)' commands and target
files, match the paths to the target files.
2. Create, rename, and move folders using unix command line tools.
3. Use grep and | to count matches of a pattern in a sample text file and save
result to another file.
4. Find what -c, -r, and -b flags do in grep by reading the manual.
5. Identify the difference in two different files using diff.
6. Open and close nano with and without saving a file.
7. Use ‘curl’ to download a file.
8. Read the variables of $PATH.
9. Explain the difference between .bash_profile and .bashrc.
10. Create a new alias by editing the .bash_profile.
11. Given a list of common scenarios, identify when it is appropriate and safe
to use sudo, and when it is a dangerous mistake.
12. Write a shell script that greets a user by their $USER name using echo.
13. Use chmod to make a shell script executable.

________________________________________________________________________________
# Navigating Your Filesystem

Imagine you're on a trip to a new country. You're carrying a dictionary, but
it's slow to translate every word you hear and you need to use a map. You would
get used to these limitations eventually, but wouldn't it be great if you spoke
a bit of the local language instead?

Believe it or not, this is likely how you've been using computers for most of
your life! Modern machines are built to make navigating easy and entertaining,
but you're not "speaking the computer's language". That changes today. Let's
explore the terminal!

We'll discuss:
- How to find your way around your files *without* double-clicking any folders,
- Creating & managing new files & directories,
- and cleaning up after yourself.

## Getting the lay of the land

You've already used a _terminal_ for some tasks like controlling Git, but let's
dive a little deeper. Your terminal is the interface you use to direct the
computer. The word "terminal", as used here, comes from the early days of modern
computing, when a _terminal interface_ (often a screen & keyboard) would be
hooked up to an otherwise manually-operated computer. This interface allowed a
human to provide instructions to the computer without turning dials or requiring
a complex manual to do so. Today, even though our terminal is built into our
computer, we still use the term to refer to the application we're using to input
our own instructions!

To keep everything in one place, we'll use the terminal that's built into Visual
Studio Code. If you haven't yet, go ahead and open it. You can do so by clicking
"View" from the top menu in VS Code, then "Terminal". You should see a new pane
at the bottom of your editor.

### Parts of the terminal

<p>
  <img src="images/cli-command-line-labeled.png" style="width: 100%; height: auto;">
</p>

1) The row in your terminal with a flashing _cursor_ is called the _command
   line_. It's exactly as the name describes: the line upon which we enter our
   commands. Because we're using the command line to instruct the computer, we
   sometimes refer to the terminal as a _command line interface_ or _CLI_.

2) The bit of text just before your command line is called the _prompt_. This
   will differ for each computer and will usually give you a little context
   about which directory you're in. You can customize your prompt to suit your
   style with custom code.

3) We refer to anything that's already been executed in the terminal as
   _output_. You'll likely see a little output in your terminal even if you
   haven't run anything yet. This is due to setup that's performed each time you
   begin a new terminal _session_.

### A few quick tricks

Here are a few keyboard shortcuts to help you along. Some of these may be review
for you. Give them a quick try before moving on.

- `Return/Enter` will submit the command you've typed.
- `↑/↓` will move up & down in your _command history_.
- Pressing `Ctrl + A` will move your cursor to the beginning of the line, while
  `Ctrl + E` will move you to the end.
- To clear the terminal screen, press `Ctrl + L`. You can still scroll up to see
  your previous output if you need to.

### Understanding directories

Let's run through a quick review of how your file system is structured. Your
computer contains both _files_ and _directories_. We distinguish these by their
content: a file contains text or binary content that we can interact with, and a
directory contains both files and other directories!

> As an aside: It's easy to confuse "files" and "folders", so it's best to use
> the term "directory" instead. It's worth an extra syllable to prevent
> confusion!

Directories and files form a tree-like structure, where each directory creates a
new branch and each file is like a leaf. We can write the _path_ to the file or
directory we want by joining all its _ancestors_ with forward slashes, like so:

```sh
  /users/app_academy_student/homework/my-homework.txt
```

Here's what that looks like in "tree" form:

<p>
  <img src="images/cli-file-system.png" style="width: 100%; height: auto;">
</p>

There are a few special short names for particular directories you should know,
too.

- `~` is your _home directory_. This is the same as `/Users/your_username/` on
  macOS.
- `/` is the _root directory_. This is the highest available directory and
  contains all the other directories & files in your file system.
- `.` is your current directory and `..` is your current directory's parent. For
  example, in `/Users/You/`, `.` refers to the `You/` directory and `..` refers
  to the `Users/` directory

## Getting around

Now, we're comfortable on our command line. Let's start navigating our
filesystem directly from our keyboard - no touchpad required!

### Where am I?

When getting started in a new place, it's often helpful to orient yourself to
your surroundings. The easiest way to orient yourself in the terminal is with
the `pwd` command. `pwd` stands for "Print Working Directory". It will print the
full path to your current directory out to your terminal. Give it a try now!

You might get back something unexpected here. If your prompt includes `~`, `pwd`
will return `/Users/your-user-name/` in its place. Remember that `pwd` always
returns the full path where you are, without any special characters or
shortcuts.

Once you know where you are, it's good to see what else is there! We can look at
what's present in the current folder with `ls`. `ls` is short for "List", and
will display the contents of whatever path you provide it with. For example, you
could run `ls .` to see your current directory's contents, or `ls ~/Projects` to
see the contents of the "Projects" directory inside your home directory. When
you don't provide any path, `ls` defaults to the contents of your current
working directory.

### A closer look at our contents

By itself, `ls` is useful but can be a little misleading. Linux & MacOS both
support the concept of hidden files. These are files or directories whose names
are preceded by a `.`. We've seen this before with the `.git` directory. Within
a Git repository, `ls` alone won't display the `.git` folder at all. We'll see
many more hidden files in upcoming lessons.

Command line instructions allow you to use _options_ to alter their behavior. We
set these options with either a single `-` (for shorthand options of one letter)
or a `--` (for option _keywords_: whole words or phrases). Here's an example
using short options with `ls`:

```sh
> ls -a -l
```

The above command runs `ls`, showing **a**ll files and displaying them in a
**l**ist format. This ensures that we see **all** files, including those that
are hidden. Viewing contents in a list format can make it a little easier to
read, and it will show us some extra info about each file/directory! We'll dig
into what that extra info means in a future lesson.

Here's one neat trick you'll see often with command line options: you can
combine short options! Instead of typing `-a` and `-l` separately, you can run
the same command this way:

```sh
> ls -al
```

Short options like this aren't order dependent, so `ls -la` will perform the
same action.

### Navigating directories

Now we know where we are and we know how to see what's around us. Let's set off
on an adventure! It's time to navigate to other directories.

We switch to a different directory with the `cd` command. `cd`, which stands for
"Change Directory", expects a path just like `ls`. Running `cd` with no
arguments will assume you'd like to change to your home directory, and you'll
end up back at `~`.

You can `cd` from any folder you have permission to access to any other folder
on your system. There's no need to move in small steps! You can jump directly
from `~` to `~/Projects/Homework-Week-1/Project-Name/code` with a single `cd`
command.

Here are some short examples of common `cd` commands you'll use:

- Change to your home directory:
  ```sh
  > cd
  # OR
  > cd ~
  # OR
  > cd /Users/YourUserName
  ```

- Navigate up a single level from your current position:
  ```sh
  > cd ..
  ```

- Move back to the last directory you were in:
  ```sh
  > cd -
  ```

### A caveat

You're browsing through your file system when you hit a snag: you encounter a
"Permission denied" error. Oh no!

Have no fear. This is perfectly normal. Your operating system has a strict
permissions system that tries its best to keep you from doing accidental damage.
This is less obvious when you're browsing folders with Finder or File Explorer,
where dangerous files/directories are hidden from view. In the terminal, though,
these unexpected blocks can be jarring.

If you have a problem with "Permission denied", it's best to ignore it and go
another direction for now. We'll discuss ways around this once you've had more
practice in the directories you already have permission to access.

 That's great, but now it's time to gain
more control of your environment. Let's discuss how to create your own files and
folders from the command line.

### Creating new files & directories

You may not have much navigating to do if you're in an empty home directory. To
start, let's discuss files. The fastest way to create an empty file is with the
`touch` command. You give `touch` a path & file name, and it creates an empty
file at that path with your given name. Here's an example:

```sh
> touch myApp.js

> touch ~/.js_settings
```

Note that `touch` doesn't put any content in the file, nor does it open the file
for editing. This is a great utility for laying your files out, but you'll
quickly want to move to a file editor (like VS Code) to make changes to these
files.

For directories, we have the `mkdir` command. `mkdir` is short for "Make
Directory" and will create a new, empty directory with the name you pass it:

```sh
> mkdir my-cool-projects

> mkdir ~/new-code
```

A common problem when learning `mkdir` is trying to create _nested_ directories.
For example, if I wanted to create a "first-week" directory inside a "homework"
directory in my home folder, I would need to ensure the "homework" directory
exists first. Here's what that looks like:

```sh
> mkdir ~/homework/first-week

mkdir: ~/homework: No such file or directory
```

We can solve this with a commonly-used short option for `mkdir`: `-p`. The `-p`
option stands for **parent**, and it will cause `mkdir` to create _all_ parent
directories it needs to create the requested directory:

```sh
> mkdir -p ~/none/of/these/directories/exist/but/now/they/will
```

One last thing: when naming files and directories, do not use spaces! You can
make multi-word names more distinct by using underscores, hyphens, and using
camelCase. While files are *allowed* to have spaces in their names, this can
complicate navigation. You'll thank yourself later if you avoid them altogether.

### Manipulating existing files

You're browsing directories. You're making files. Woohoo!

Just like using your mouse in Finder, you can copy/relocate files and
directories from the terminal. The commands you'll need are `cp` and `mv`.

`cp` is short for "copy" and will create a duplicate of a file. It requires two
arguments: a _source_ and a _destination_. _source_ can be the relative or
absolute path of a file, _destination_ can be a path to a file or a directory.
If _destination_ is a directory, `cp` will copy the source file into that
directory. If _destination_ is a file path, `cp` will copy the source file into
that new location.

**Gotcha**: If a file already exists in the destination of your copy command,
`cp` will overwrite the existing file.

```sh
# Will copy the file into the `people` subdirectory.
> cp best-friend.txt people/

# This command is identical to the above.
> cp best-friend.txt people/best-friend.txt
```

In each of these cases, we'll create an exact copy of `best-friend.txt` from
your current working directory and place that copy in the `people` folder.

You can copy directories just like files, but you'll need a special short
options to do so: `-r`. This option, short for "Recursive", copies not just the
directory but all of its contents! Without it, the directory will fail to copy:

```sh
> ls
my_dir my_other_dir

> cp my_dir my_other_dir/
cp: -r not specified; omitting directory 'my_dir/'
```

Alternatively, `mv` "moves" a file from one place to another. Think of this like
the "Cut" options on other operating systems. Again, you pass two arguments:

```sh
> mv breakfast-foods/cereal.txt anytime-foods/cereal.txt

# or identically:
> my breakfast-foods/cereal.txt anytime-foods/
```

What if you need to rename a file? There is no `rename` command in the terminal.
Instead, you'll use `mv` to accomplish this:

```sh
mv speled-rong.txt spelled-wrong.txt
```

Like `cp`, `mv` can be used to move or rename directories. However, unlike `cp`,
`mv` does *not* require a flag to do so. This is because the `mv` operation
simply renames the directory, so we're not concerned about the contents within
it.

### Clean up: aisle `~`!

Okay, file system traveler. You've thrown `mkdir` and `touch` around for long
enough! Let's discuss how we remove files and directories.

There are two removal commands in your terminal: `rm` and `rmdir`. The former is
for files or directories, while the latter is for directories only. The use
cases can be a little confusing, so let's look at some examples.

First, `rmdir`. This command, short for "Remove Directory" is only for removing
an empty directory. If the directory has any files or other directories within
it, the command will fail. You'll use this command occasionally for cleaning up
extra directories you've created, but you're more likely to use the other
removal command we mentioned.

```sh
# Remember, ~/my-app must be empty for this succeed!
rmdir ~/my-app
```

Last, `rm`. This command is short for "Remove" and can be one of the most
dangerous tools in your arsenal. We've mentioned this before, but it bears
repeating: *never use `rm` unless you're absolutely sure of what you're
removing*! The terminal is often much less forgiving than the Finder app or
Recycle Bin.

To use `rm`, you provide a filename or path. If you need to remove a directory
along with all of its contents, you can use the `-r` short option, which will
"Recursively" remove all files within the directory before removing the
directory itself. Your terminal will guide you along this process - all you have
to do is type "y" for "Yes" or any other key for "no" as it asks you about each
file within the directory you're deleting. Once a file has been `rm`'ed, it's
unrecoverable, so be *careful* about what you use `rm` on!

```sh
> rm file-we-dont-need.txt

> rm -r directory-full-of-files-we-dont-need/
```

Try practicing these new tools by cleaning up the mess you've made while
experimenting. You'll get lots more practice using these as the days progress.

## What we've learned

Navigating in the terminal is a little different than we're used to, but it's
much faster to type commands than to drag a mouse! You should now have a greater
mastery of:
- Exploring your file system via the terminal,
- Creating, updating, and removing files and directories,
- Using command line options to enhance our tools with new functionality.

________________________________________________________________________________
# Common Tasks On The Command Line

Let's go for a deeper dive into the tools we'll use day in and day out. We'll
cover:

- Searching for files by content,
- Performing multiple commands in sequence,
- Terminal text editors & file comparison,
- and how to perform common web tasks from the command line.

## `grep` marks the spot

One of the most common tasks you'll have is searching for a particular piece of
code in a project. This might be to help diagnose unexpected behavior, or it
might just be because you can't remember exactly where you left off! Either way,
having a tool to help you find your way to a specific point in your code is
critical to your workflow.

`grep` is a command line utility that was originally created in 1974 and stands
out as a well-tested, reliable tool that does one thing well: text search. The
name comes from the command sequence `g/re/p`, meaning "**G**lobally search for
a **R**egular **E**xpression and **P**rint" (We'll discuss _regular expressions_
in a future lesson). You can use `grep` to find text in a particular file or
across multiple files in a particular directory! It's like using "Cmd + F" from
the CLI.

The simplest use of `grep` is with the contents of a single file. Here's an
example of using it to find all the variables in a JavaScript file:

```sh
> grep var ./myAppFile.js
```

Notice that `grep` expects at least 2 arguments: a _pattern_ and a _source_ to
search. In the above example, `var` is the pattern and `./myAppFile.js` is the
source. If your search string is more than a single word, you'll need to wrap it
in double-quotes:

```sh
> grep "I'm a programmer" ./resume.txt
```

By default, `grep` will return the whole line where your search string appears.
This can be a little confusing at first:

<p>
  <img src="images/cli-raw-grep-output.png" style="width: 100%; height: auto;">
</p>

In this example, there are four lines where the word "remote" appears in the
file `.git/config`. That's not particularly intuitive from this raw output,
though! Let's look at some ways to provide more helpful info from `grep`.

### Common `grep` options

We'll occasionally need to modify `grep`'s default behavior. One common
situation is when searching directories. It's likely that you'll want to use
`grep` to find any files in a directory that contain a certain pattern. You can
do this with the `-r` option, which stands for "**R**escursive". When run this
way, `grep` expects a directory path as its source. It will search the directory
and all of its children (files and subdirectories). Be aware: if there are lots
of files and you're searching a common phrase, you might get back more than you
expect with this option!

Another commonly-used option is `-n`, for "line **N**umber". This will show you
the exact line for each match. Handy if you want to find something extra-fast!

By default, `grep` is case-sensitive, so searching for `Let` won't bring back
instances of `let`. To override this behavior, use the `-i` option, for
"**I**gnore Case".

The last common option we'll discuss is `-c`, for "**C**ount". This will return
only the number of matches, and not a full list of them. If you use this option
in conjunction with `-r`, you'll see filenames for each count as well. This will
be helpful when trying to track trends in code or when you need to know which
directory contains the largest number of matches.

## Teach yourself anything

`grep`, like most terminal commands, has many more options than we've discussed.
How can we keep track of them all? Fortunately, we don't have to! Our system
includes the `man` utility (short for **Man**ual) to help guide us when
questions arise.

To learn more about any built-in command, just use `man`:

```sh
> man grep
```

This will open `grep`'s manual page, the official documentation for the command.
You'll open in a _pager_, a lightweight document viewer meant to run in your
terminal. To browse the `man` page, use your arrow keys to scroll up & down, and
press the "Q" key on your keyboard to **q**uit.

`man` pages contain all the info you need to work with a command line utility.
They're typically structured in a predictable way:

- A short summary & description of special features at the top,
- Command line options with explanations in the middle,
- And examples & cool facts (like when the command was created) at the bottom.

They are often terse & technical, but `man` pages are the official word on the
tool you're curious about and are always a good place to start. Take some time
to read through the `man` pages of some of the commands we've already covered!

## Command redirection

With both `man` and `grep`, we experienced some serious terminal overload.
Wouldn't it be great if we could send that output somewhere else, like our text
editor or a file to read later? We can do this via _command redirection_.

As the name implies, redirection is all about taking the output from one command
and making it the input for a different command. Let's look at a very simple
example using `|`, the _pipe operator_, where we'll combine `man` and `grep` to
discover how to count pattern matches on a `man` page:

```sh
> man grep | grep -C1 count
```

Notice that we're not using the letters "L" or "I" but the vertical pipe
character, found on the same key as `\` and above your "Return" key on US
keyboard layouts. This operator takes the output from its left side and passes
it to the command on its right as the final argument.

When we run the command above, we'll get back only the lines from `grep`'s `man`
page that include the word "count", along with one line above & below for
context:

<p>
  <img src="images/cli-grep-redirected-output.png" style="width: 100%; height:
  auto;">
</p>

It's common to want to save output from a command into a file, too! We can do
this a few different ways. The easiest is to use the `>` or `>>` operators:

```sh
# The single > operator will create a new file
# to place output in. Existing content will be overwritten.
> grep -r "TODO" my_app/ > my-app-todos.txt

# The double > operator will append your output
# to an existing file (or create a new one if needed).
> grep "my-name" list-of-names.html >> name-locations.txt
```

These redirection operators are lightning-quick, but have the caveat of not
showing you the output before writing it to the given file! We can use the `tee`
utility to both see our output and have it written to a file:

```sh
> ls my_directory/ | tee directory_contents.txt
```

Your command line supports its own scripting language and we're just scratching
the surface with these redirection operators. Whenever you find yourself
performing one or two simple tasks in the same order numerous times, consider
how you might use redirection to simplify that process.

## Editing files directly from the CLI

We can create files from the command line, search for content, and even combine
commands into a single line! What about editing files directly from the command
line too? Yup - there's a utility for that!

> In fact, there are a **large** number of text editors available for the
> command line. Two of the most popular ones you'll hear about are _vim_ and
> _emacs_, which can both act as full development environments with a little
> customization! While you'll see lots of details about these editors online,
> it's best to avoid them for now. We'll focus on simpler editors with a much
> smaller learning curve.

We're going to take a look at `nano`, a terminal text editor that provides
easy-to-read instructions and is available on most systems. To get started, just
type `nano` on your command line.

Let's take a look at the `nano` interface:

<p>
  <img src="images/cli-nano-diagram.png" style="width: 100%; height: auto;">
</p>

The upper part of `nano` is the editor body. Here you can type just like you
would in any other editor. Text and linebreaks will work as expected, but
support may be limited for fancy characters or keyboard shortcuts.

At the bottom of the screen you can see the command menu. These are the actions
available to you and the keystrokes you need to access them. Remember that `^`
represents "Ctrl" on your keyboard, so `^O` is the same as `Ctrl + O`.

When you attempt to save a file via `Ctrl + O`, you may be asked to confirm the
filename. Type whatever name you'd like your file to have and hit "Enter" to
save. Don't worry: if you've forgotten to save recently, `nano` will help you
out by asking if you want to save unsaved changes before exiting!

When you run `nano` with no arguments, it opens to a new _buffer_, or empty
space in memory ready for you to write. No files will be created until you save
them. You can also run `nano` with a file path as an argument. `nano` will open
the given file for you to edit:

```sh
> nano myApp.js
```

### Why a terminal editor?

While `nano` is pretty stripped-down compared to a tool like VS Code, there are
really great reasons to get familiar with it. The biggest benefits come when
working in remote environments.

You'll sometimes need to log into a server somewhere else in the world and
change a configuration file or run an update. If you need to edit text in that
remote environment, it's unlikely you'll have access to VS Code. It's
dramatically more likely that you'll have access to `nano` on your remote
terminal. Now you can confidently edit files on computers you may never see in
person! How cool is that?

## Bringing the internet into your terminal

One last task you'll perform frequently on the command line is downloading
files. This could be anything: an icon pack for your cool new app, an
installation script for a larger program, or even a whole webpage to scrape for
a project.

You can use the `curl` command to download from a URL to a file on your
computer:

```sh
> curl https://www.my-website.com/my-file.html
```

With no other options, `curl` would download the contents of that URL to a new
file titled `my-file.html` on our system. If you'd rather name the new file
yourself, you can use the `-o` option:

```sh
> curl -o my-downloaded-file.html https://www.my-website.com/my-file.html
```

Like our other CLI utilities, `curl` is well-known and highly available. There's
an extensive `man` page that's worth browsing through, too! `curl` offers lots
of options that let you manage authentication, upload your own files, or even
customize the type of request you're making. It's a powerful tool behind its
rather simple facade.

## What we've learned

Whew! If this feels like a lot of new tools, don't worry! They're all things
you'll use on a daily basis. The beauty of using terminal tools like these is
their simplicity: you can build complex workflows out of very simple utilities,
and you get lots of practice in the process.

After reading this lesson and exploring on your own, you should be able to:
- find specific phrases in files and directories with `grep`,
- learn more about any command using `man`
- link commands together by redirecting output as input,
- edit file content using `nano`,
- and download files from the web using `curl`.

________________________________________________________________________________
# Understanding The Shell

As a developer, you'll spend much of your time working at the command line.
Learning the commands you can use is important, but it's even more important to
build a foundational understanding of where those commands go! Let's get under
the hood and discuss the shell.

You'll learn:

- What a "shell" is,
- Shell-specific files and how to customize them,
- How commands are executed from the command line.

## No turtles here!

We've used the word "shell" a few times with no context. Let's fix that now! A
_shell_ in computing terms is the software layer that translates your command
line instructions into actual commands. Generally speaking, the shell serves two
purposes: **Calling applications using your input** and **supporting user
environments/customization**.

The shell is a small part of the much larger _operating system_, the software
that sits between your input and the microchips inside your computer. The
operating system, or _OS_, translates your actions all the way down into machine
instructions that can be processed by your CPU. You've heard of the most popular
OSes out there: Windows, macOS, and Linux.

### Selling C shells by the seashore

Like most things in web development, many people have strong opinions about
which shell is best. There are many shells available that you can install & use,
each with its own idiosyncrasies.

In our lessons, we'll focus on two shells, the _Bash_ shell and _Zsh_.
Bash has been around for a little over 30 years and has been battle-tested on
the most popular operating systems on the web. The biggest Linux operating
systems all use it as the default shell, and macOS has used it as the default
until switching to [Zsh in macoS Catalina].

Zsh is now the default shell in macOS Catalina and has been in use since the
1990s and has a strong following amongst linux users.

You'll want to keep which shell you use in mind as you search for help with
command line problems, as an answer intended for one shell may not work with
your shell.

One nice thing about Zsh is it's scripting compatibility with Bash. Which means
a shell script written for Bash will work in Zsh, although the opposite is not
always true.

### Shells vs Terminals

Your operating system may have an application called a "Terminal".  It's good to
note that a terminal is not a shell and a shell is not a terminal.  Terminal
applications are really emulating a piece of hardware known as a _terminal_ which
nobody really uses anymore.

![Terminal]

So we emulate a terminal using an application on our computers.  The Terminal
application will then execute the shell and it will give us a prompt so we can
then type commands.

### Shell Prompts

You'll notice your shell prompts you to type something by putting either a `$`
or a `%` before the cursor.  Bash uses `$` by default while Zsh uses a `%`. But
there's one more character you may see as your prompt character and that is
`#`. This appears if you are logged in as `root`, the unix superuser. Since with
great power comes great responsibility, whenever we see the `#` prompt we want
to be careful what we type because we could delete system files or cause files
to have the wrong permissions if we aren't absolutely sure what we are doing.

### Two purposes

A shell's primary purpose is to interpret your commands. It does this by looking
for applications installed on your computer and sharing any arguments or
environment-specific data they need. Let's think about the following command,
which will print the word "Test" to your terminal:

```sh
$ echo Test
```

In this command, `echo` refers to a program called "echo", while `Test` is an
argument you've given the program. Bash doesn't know what `echo` does, only that
it needs to find an application with that name and give it the word "Test". The
shell trusts the application will handle this input appropriately.

Bash searches for applications using the `PATH` variable. `PATH` is a special
variable that's available system-wide and includes all the directories you might
store applications in. You can view your own `PATH` using that `echo` utility:

```sh
$ echo $PATH
```

You should see a list of directory paths separated by colons. When you run a
command, your shell extracts the application name from the command and starts
going through the `PATH`, directory by directory, looking for that name. Once it
finds it, it passes along your input (or just starts the application, if there's
no input available), and stops looking.

Note that we didn't just write `echo PATH` above. Instead, we included a `$`
before the name of the variable. That's not a typo! On the command line, `$`
before a word indicates that we're looking for a variable with the following
name. Executing the command `echo PATH` would simply print out the word "PATH".

You might see a risk from this process right away: what happens if we have two
different versions of the same program? Bash will call whichever version of the
program it finds first in your `PATH`! If you're unsure which version of an
application Bash is going to run, you can use the `which` command:

```sh
$ which echo
/bin/echo
```

This output means Bash is going to run the application found at `/bin/echo`
every time you enter a command beginning with "echo". If `which` returns a
different version than you'd like to be running, you'll need to modify your
`PATH`, which we'll discuss as part of our customization options.

Speaking of customizations: that's the shell's secondary purpose! Your shell
makes things uniquely yours in a few different ways. The first is _environment
variables_. These are variables that are stored in memory and made available to
running applications by the shell. `PATH` is an example of an environment
variable. Another is `HOME`, which stores the location of your home directory.
You can see all your environment variables with the `env` command, but be
prepared to scroll! The list can get pretty long.

Other customizations include scripts, command aliases, and your prompt. We'll
discuss these more as we dig into deeper customization options in this and
future lessons.

### From the command line to the screen

It can be helpful to have a high-level overview of what's happening after you
press "Enter" on the command line. Check out the diagram below to see how your
command goes from keyboard to the monitor.

<p>
  <img src="https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-shell-breakdown-hc.png" style="width: 100%; height: auto;">
</p>

There are lots of extra steps we're overlooking here, but this should help you
visualize the role of the shell.

## Customizing your environment

Your shell's defaults (which may differ from system to system) are likely not
doing a lot to help you. They're meant to get things up and going quickly, but
you'll want to expand on them to suit your own tastes! Let's talk about how to
make changes to your shell.

### Startup files

The easiest way to make changes to the shell is directly from the command line.
For instance, you can use the `export` command to change/initialize a new
environment variable:

```sh
$ export NEW_VARIABLE=value
```

However, if you close your current terminal or open a new one, your environment
variable will no longer be present! To persist environment variables and other
customization settings, you'll need to put them in a file.

Bash supports several files intended for you to customize: `.profile`,
`.bash_profile` and `.bashrc`.

Zsh supports several customization files as well, but we will only need to use
`.zshrc` for most setups.

Each of these customization files are found in your home directory and are
hidden (as indicated by the `.` at the beginning of filename).

These customization files are sometimes referred to as _dotfiles_.

These startup files are executed automatically at different times when
you start your shell.

### Bash startup files

The `.bash_profile` or `.profile` are executed when bash is started with a
`-l` or `--login` command line flag.  This is called a _login shell_.  If you
run bash without this commmand line flag it instead is a _non-login shell_ and
runs the `.bashrc` file.

To add a little complexity, Bash will run the `.profile` only if there isn't a
`bash_profile`.

Now to confuse matters even more, often the `.bash_profile` will have a snippet of
code in it that executes the `.bashrc` as well (this is the default on Ubuntu
for instance)

To read more about which files Bash runs at which times at this link:
https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html

### Zsh startup files

The `.zshrc` file is started anytime you start Zsh for both _login shells_ and
_non-login_ shells. It does have a `.zlogin` file which is only executed on
login shells but since `.zshrc` loads on both most Zsh users don't use `.zlogin`
and instead put all their customizations in `.zshrc`.

To read more about all Zsh's various startup files you can follow this link:
http://zsh.sourceforge.net/Intro/intro_3.html

## To Login or Not to Login

So how can you tell if you have a login shell or a non-login shell? It depends
on how your operating system works.

### macOS

On macOS prior to Catalina if you open the Terminal
application, it runs Bash as a _login shell_ and therefore runs the
`.bash_profile` on every terminal window you open.

After macOS Catalina, if you open the Terminal application it launches Zsh
as a _login shell_ and therefore runs both `.zlogin` and `.zshrc`.

### Windows using WSL

On Windows using WSL, the Windows Ubuntu terminal app runs every bash
as a _login-shell_ and therefore will run the `.profile`.

### Ubuntu Linux

On Ubuntu Linux, by default the `.profile` is executed when
you login to the Ubuntu Desktop. Ubuntu does not include `.bash_profile` by
default. Then when you open the Terminal application, it runs each bash as a
_non-login shell_ which only runs `.bashrc`.

This can be somewhat frustrating to have to logout of the Ubuntu Desktop and
login back in each time you change your startup file. So instead you can change
the Terminal application to run each shell as a login shell by checking this
checkbox in the Terminal preferences that reads _"Run command as a login shell"_

![Ubuntu Terminal]

### Visual Studio Code

When you use Visual Studio Code's integrated terminal, on macOS it defaults
to being a login shell, while on Linux and Windows it does not.

You can change the integrated shell in vscode on Ubuntu and Windows using WSL
by modifying or adding these settings in the VSCode `settings.json` file.

```json
  "terminal.integrated.shell.linux": "bash",
  "terminal.integrated.shellArgs.linux": [ "-l" ]
```

If you use macOS Catalina, VSCode will still default to bash in it's integrated
shell, so you can set it to use Zsh like so:

```json
  "terminal.integrated.shell.osx": "zsh",
  "terminal.integrated.shellArgs.osx": [ "-l" ]
```

### Whew this seems complicated, can you just tell me which file to use?

Sure, here's our recommendations on how to customize your system depending on
platform to have the best results.

#### macOS Catalina with Zsh

- Put your customizations into your `.zshrc` file.
- Change Visual Studio Code to use `zsh` as it's shell.

#### macOS pre-Catalina with Bash

- Put your customizations into your `.bash_profile` file.

#### Ubuntu Linux

- Change your terminal to open login shells
- Put your customization into your `.profile` file.
- Change Visual Studio Code's integrated terminal to launch login shells as well.

#### Windows with WSL

- Put your customization into your `.profile` file.
- Change Visual Studio Code's integrated terminal to launch login shells as well.

### Customization options

Whew, that's a lot of new jargon & theory! Let's look at some practical examples
of what you can do with your dotfiles in Bash or Zsh.

Adding new options to your dotfiles starts with editing them.

You can open them directly from the command line using VS Code:

```sh
$ code ~/.bash_profile
# OR
$ code ~/.bashrc
# OR for Zsh
% code ~/.zshrc
```

Once you've saved your changes, you'll need to load any updated files into your
shell. You can do ths with the `source` command. `source` will execute the file
it's given, updating your currently-running environment with any changes you've
made:

```sh
# For Bash
$ source ~/.bash_profile
```

OR

```sh
# For Zsh
% source ~/.zshrc
```

One of the most common customizations are environment variables. These should
always be capitalized & use underscores instead of spaces to delineate words.
Here are a couple examples of code customizing environment variables, along with
comments explaining how each works.

```sh
# The simplest option: adding a totally new environment variable.
export FAVORITE_COLOR=blue

# Let's overwrite an existing environment variable with our own.
export HOME=/User/Student/Home

# Time to get get more creative: what if we want to _prepend_
# to the PATH variable, instead of overwriting it if it exists?
export PATH=/User/Student/Applications:$PATH
```

In that last example, notice how we used `$` before `PATH`. The dollar sign
indicates that we're referencing a variable. Bash will replace `$PATH` with the
value of the `PATH` variable, so we're effectively adding our own directory to
the beginning of the `PATH` variable. You'll see this technique used a **lot**
in dotfiles.

Another common customization is aliasing. An _alias_ is a shorthand way of
running a command. You might alias because a command is very long to type, or to
modify the system's default behavior! Here are some examples:

```sh
# Here's a Git alias that will save you a few keystrokes.
alias gst="git status"

# Some more Git magic: show the short log with an even shorter command!
alias glog="git log --oneline"

# By default, 'rm' will remove the file you pass it. The '-i'
# option makes 'rm' ask you "Are you sure?" before removing the
# given file. This is a great safety net to have while you're learning!
alias rm="rm -i"
```

Don't forget to `source` your dotfiles if you're following along!

Entering an alias on the command line makes the aliased command act just like
the full command was entered. You can still pass arguments like you normally
would! Here are some "before & after" examples of our aliases in action:

<p>
  <img src="https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-aliases-gst.png" style="width: 100%; height: auto;">
</p>

Notice how we get the same output from `gst` as `git status` with significantly
less typing.

<p>
  <img src="https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-aliases-glog.png" style="width: 100%; height: auto;">
</p>

Our `glog` alias provides us with a short, quick-to-read commit history that
takes even fewer characters typed than the default `git log`!

<p>
  <img src="https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-aliases-rm.png" style="width: 100%; height: auto;">
</p>

The `rm` alias is an example of changing the default behavior of a command.
Before we aliased `rm`, removing a file happened with no confirmation at all.
After the alias is applied, we see a prompt that lets us respond "y" (for
"**y**es") or "n" (for "**n**o"). Think of this as the command line version of a
"Confirm" or "Cancel" pop-up window!

## What we've learned

We're diving deeper behind the scenes of our terminal & operating system, and
hopefully you're beginning to understand how things work together! You should
now be comfortable with:

- What a "shell" is in computer terms,
- explaining, at a very high level, how your computer interprets your commands,
- reading and redefining the `PATH` environment variable,
- and customizing your system via _dotfiles_

[prior to Catalina which uses Zsh]: https://support.apple.com/en-us/HT208050
[Zsh in macoS Catalina]: https://support.apple.com/en-us/HT208050
[Ubuntu Terminal]:https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-shell-basics-ubuntu-terminal.png
[Terminal]:https://upload.wikimedia.org/wikipedia/commons/thumb/9/9f/DEC_VT100_terminal_transparent.png/675px-DEC_VT100_terminal_transparent.png at the command line.
Learning the commands you can use is important, but it's even more important to
build a foundational understanding of where those commands go! Let's get under
the hood and discuss the shell.

You'll learn:

- What a "shell" is,
- Shell-specific files and how to customize them,
- How commands are executed from the command line.

## No turtles here!

We've used the word "shell" a few times with no context. Let's fix that now! A
_shell_ in computing terms is the software layer that translates your command
line instructions into actual commands. Generally speaking, the shell serves two
purposes: **Calling applications using your input** and **supporting user
environments/customization**.

The shell is a small part of the much larger _operating system_, the software
that sits between your input and the microchips inside your computer. The
operating system, or _OS_, translates your actions all the way down into machine
instructions that can be processed by your CPU. You've heard of the most popular
OSes out there: Windows, macOS, and Linux.

### Selling C shells by the seashore

Like most things in web development, many people have strong opinions about
which shell is best. There are many shells available that you can install & use,
each with its own idiosyncrasies.

In our lessons, we'll focus on two shells, the _Bash_ shell and _Zsh_.
Bash has been around for a little over 30 years and has been battle-tested on
the most popular operating systems on the web. The biggest Linux operating
systems all use it as the default shell, and macOS has used it as the default
until switching to [Zsh in macoS Catalina].

Zsh is now the default shell in macOS Catalina and has been in use since the
1990s and has a strong following amongst linux users.

You'll want to keep which shell you use in mind as you search for help with
command line problems, as an answer intended for one shell may not work with
your shell.

One nice thing about Zsh is it's scripting compatibility with Bash. Which means
a shell script written for Bash will work in Zsh, although the opposite is not
always true.

### Shells vs Terminals

Your operating system may have an application called a "Terminal".  It's good to
note that a terminal is not a shell and a shell is not a terminal.  Terminal
applications are really emulating a piece of hardware known as a _terminal_ which
nobody really uses anymore.

![Terminal]

So we emulate a terminal using an application on our computers.  The Terminal
application will then execute the shell and it will give us a prompt so we can
then type commands.

### Shell Prompts

You'll notice your shell prompts you to type something by putting either a `$`
or a `%` before the cursor.  Bash uses `$` by default while Zsh uses a `%`. But
there's one more character you may see as your prompt character and that is
`#`. This appears if you are logged in as `root`, the unix superuser. Since with
great power comes great responsibility, whenever we see the `#` prompt we want
to be careful what we type because we could delete system files or cause files
to have the wrong permissions if we aren't absolutely sure what we are doing.

### Two purposes

A shell's primary purpose is to interpret your commands. It does this by looking
for applications installed on your computer and sharing any arguments or
environment-specific data they need. Let's think about the following command,
which will print the word "Test" to your terminal:

```sh
$ echo Test
```

In this command, `echo` refers to a program called "echo", while `Test` is an
argument you've given the program. Bash doesn't know what `echo` does, only that
it needs to find an application with that name and give it the word "Test". The
shell trusts the application will handle this input appropriately.

Bash searches for applications using the `PATH` variable. `PATH` is a special
variable that's available system-wide and includes all the directories you might
store applications in. You can view your own `PATH` using that `echo` utility:

```sh
$ echo $PATH
```

You should see a list of directory paths separated by colons. When you run a
command, your shell extracts the application name from the command and starts
going through the `PATH`, directory by directory, looking for that name. Once it
finds it, it passes along your input (or just starts the application, if there's
no input available), and stops looking.

Note that we didn't just write `echo PATH` above. Instead, we included a `$`
before the name of the variable. That's not a typo! On the command line, `$`
before a word indicates that we're looking for a variable with the following
name. Executing the command `echo PATH` would simply print out the word "PATH".

You might see a risk from this process right away: what happens if we have two
different versions of the same program? Bash will call whichever version of the
program it finds first in your `PATH`! If you're unsure which version of an
application Bash is going to run, you can use the `which` command:

```sh
$ which echo
/bin/echo
```

This output means Bash is going to run the application found at `/bin/echo`
every time you enter a command beginning with "echo". If `which` returns a
different version than you'd like to be running, you'll need to modify your
`PATH`, which we'll discuss as part of our customization options.

Speaking of customizations: that's the shell's secondary purpose! Your shell
makes things uniquely yours in a few different ways. The first is _environment
variables_. These are variables that are stored in memory and made available to
running applications by the shell. `PATH` is an example of an environment
variable. Another is `HOME`, which stores the location of your home directory.
You can see all your environment variables with the `env` command, but be
prepared to scroll! The list can get pretty long.

Other customizations include scripts, command aliases, and your prompt. We'll
discuss these more as we dig into deeper customization options in this and
future lessons.

### From the command line to the screen

It can be helpful to have a high-level overview of what's happening after you
press "Enter" on the command line. Check out the diagram below to see how your
command goes from keyboard to the monitor.

<p>
  <img src="https://-open-assets.s3-us-west-1.amazonaws.com/Module-Unix/cli/assets/cli-shell-breakdown-hc.png" style="width: 100%; height: auto;">
</p>

There are lots of extra steps we're overlooking here, but this should help you
visualize the role of the shell.

## Customizing your environment

Your shell's defaults (which may differ from system to system) are likely not
doing a lot to help you. They're meant to get things up and going quickly, but
you'll want to expand on them to suit your own tastes! Let's talk about how to
make changes to your shell.

### Startup files

The easiest way to make changes to the shell is directly from the command line.
For instance, you can use the `export` command to change/initialize a new
environment variable:

```sh
$ export NEW_VARIABLE=value
```

However, if you close your current terminal or open a new one, your environment
variable will no longer be present! To persist environment variables and other
customization settings, you'll need to put them in a file.

Bash supports several files intended for you to customize: `.profile`,
`.bash_profile` and `.bashrc`.

Zsh supports several customization files as well, but we will only need to use
`.zshrc` for most setups.

Each of these customization files are found in your home directory and are
hidden (as indicated by the `.` at the beginning of filename).

These customization files are sometimes referred to as _dotfiles_.

These startup files are executed automatically at different times when
you start your shell.

### Bash startup files

The `.bash_profile` or `.profile` are executed when bash is started with a
`-l` or `--login` command line flag.  This is called a _login shell_.  If you
run bash without this commmand line flag it instead is a _non-login shell_ and
runs the `.bashrc` file.

To add a little complexity, Bash will run the `.profile` only if there isn't a
`bash_profile`.

Now to confuse matters even more, often the `.bash_profile` will have a snippet of
code in it that executes the `.bashrc` as well (this is the default on Ubuntu
for instance)

license: MIT

branch_name: refs/heads/master

extension: md

gha_license_id: MIT

license_type: permissive