Compare commits

..
3 Commits
Author SHA1 Message Date
Florian WaltherandClaude Opus 5.5 22ce8c00d4 add assignments name=value and unset
- name=value (one or more) without a command sets variables of the shell
- name=value before a command sets them only in its environment (for
  builtins only while they run)
- an assignment needs an unquoted name and =; the value is not split
  and not expanded as wildcard or brace pattern, ~ is expanded after =
  and :
- the status of an assignment is the one of its last $(...)
- new builtin unset

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 00:17:38 +02:00
Florian Walther fb94e98da5 updated README 2026-10-04 00:16:29 +02:00
Florian WaltherandClaude Opus 5.5 f79fb0b8d5 add control structures: if, for, while, until
- new parser (parser.go) for lists, pipelines, ! and if/elif/else/fi,
  for ... in ... do/done, while/until ... do/done; simple commands are
  kept as text and expanded when they run (in every iteration)
- break [n], continue [n] and the : builtin
- incomplete input (open control structure, trailing &&, |, quote or
  backslash) continues on the next line: prompt "> " in interactive
  shells, scripts and startup files read on
- Ctrl+C aborts the whole command line, including loops
- aliases are expanded at the start of every simple command, also inside
  control structures
- control structures in pipelines, with redirections or in the
  background are reported as not supported
- completion: commands after if, then, do etc.
- main_test.go reformatted with gofmt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 00:05:28 +02:00
9 changed files with 1597 additions and 230 deletions

No files matched your search

+80 -3
View File
@@ -1,8 +1,8 @@
`gosh` eine einfache Shell, geschrieben in golang.
`gosh` eine Shell, in golang geschrieben.
```
gosh - Go Shell
Version: 0.2.1 (branch: main, commit: 03e791c)
Version: 0.13.0 (branch: main, commit: f79fb0b)
Built-in commands:
cd [dir] - Change directory (cd with no args goes to home)
@@ -12,6 +12,7 @@ Built-in commands:
quit [code] - Quit shell with status code (default: $?)
set VAR=val - Set environment variable
export VAR=val - Export environment variable
unset VAR - Remove a variable
source file - Run the commands of file in this shell (also: . file)
jobs - List background and stopped jobs
fg [%n] - Continue job n (default: current job) in the foreground
@@ -19,6 +20,9 @@ Built-in commands:
wait [%n] - Wait for job n or all background jobs
alias [name[=value] ...] - Define or show aliases (alias ll='ls -l')
unalias name - Remove an alias (unalias -a: all aliases)
: - Do nothing (exit status 0)
break [n] - Leave the innermost (or n-th) for/while/until loop
continue [n] - Continue with the next iteration of a loop
help - Show this help
Features:
@@ -41,6 +45,9 @@ Features:
GOSH_COLORS='user=1;33:path=36' (keys user, root, path, error; values
are ANSI SGR codes, empty = no color); disabled with $NO_COLOR,
TERM=dumb or when output is not a terminal
- Variables: name=value sets a variable (like set), name=value command
sets it only for that command; the value is not split or expanded as
a pattern
- Environment variable expansion ($VAR, ${VAR}, ${VAR:-default});
using an unset variable is an error, use ${VAR:-} for an empty default
- Command substitution: $(command) is replaced by the output of command
@@ -48,6 +55,9 @@ Features:
stays one word
- Command lists: a ; b (run both), a && b (b only if a succeeded),
a || b (b only if a failed); $? is the exit status of the last command
- Control structures: if cmd; then ...; elif cmd; then ...; else ...; fi,
for x in words; do ...; done, while cmd; do ...; done, until ...,
! cmd negates the status; they can span several lines (prompt "> ")
- Pipes (a | b) and redirections: < file, > file, >> file, 2> file,
2>> file, 2>&1, >&2, &> file (stdout and stderr), &>> file
- Job control: command & runs in the background, Ctrl+Z stops the
@@ -125,6 +135,73 @@ export LANG=de_DE.UTF-8
export PATH=$HOME/bin:$PATH
```
### Kontrollstrukturen
gosh kennt `if`, `for`, `while` und `until` wie `sh`, auf einer Zeile oder über mehrere Zeilen:
```bash
if [ -f ~/.goshrc ]; then
echo "rc vorhanden"
elif [ -d ~/.config ]; then
echo "nur .config"
else
echo "nichts"
fi
for f in *.log; do
gzip "$f"
done
for i in {1..5}; do echo "Durchlauf $i"; done
n=3
while [ $n -gt 0 ]; do
echo $n
n=$(expr $n - 1)
done
until ping -c1 -W1 server >/dev/null; do sleep 1; done
```
| Konstrukt | Bedeutung |
|---------------------------------------------|-------------------------------------------------------------------|
| `if befehl; then …; elif …; then …; else …; fi` | führt den Zweig aus, dessen Bedingung den Exit-Status 0 hat |
| `for x in wörter; do …; done` | setzt `x` nacheinander auf jedes Wort |
| `while befehl; do …; done` | wiederholt, solange `befehl` erfolgreich ist |
| `until befehl; do …; done` | wiederholt, bis `befehl` erfolgreich ist |
| `! befehl` | kehrt den Exit-Status um |
| `break [n]`, `continue [n]` | verlässt die (n-te) Schleife bzw. springt zum nächsten Durchlauf |
| `:` | tut nichts, Exit-Status 0 (z. B. `while :; do …; done`) |
- Bedingungen sind normale Befehle (auch Listen mit `&&`/`||`), meist `test` bzw. `[ … ]` (z. B. `[ -f datei ]`, `[ "$a" = b ]`, `[ $n -lt 3 ]`).
- Die Wörter nach `for … in` werden wie Argumente expandiert: `*.go`, `{1..10}`, `$VAR`, `$(befehl)`. Ohne `in` läuft die Schleife nicht (Positionsparameter gibt es nicht).
- Variablen, `$(...)` und Wildcards im Rumpf werden bei jedem Durchlauf neu ersetzt.
- Ist ein Befehl noch nicht vollständig (z. B. nach `if …; then` oder `for …` + Enter, nach `&&`, `|` oder einem `\` am Zeilenende), zeigt gosh den Fortsetzungs-Prompt `> `; Ctrl+C verwirft die angefangene Eingabe. Skripte und Startdateien lesen entsprechend weiter.
- Ctrl+C bricht die ganze Befehlszeile ab, also auch eine Schleife (`while true; do sleep 1; done`).
- Der Exit-Status ist der des zuletzt ausgeführten Befehls; ein `if` ohne passenden Zweig und eine Schleife ohne Durchlauf haben den Status 0.
- Nicht unterstützt: Kontrollstrukturen in Pipes (`for …; done | sort`), mit Umleitung (`done > datei`) oder im Hintergrund (`done &`), außerdem `case`, Funktionen und `read`. Arithmetik geht z. B. mit `n=$(expr $n + 1)`.
### Variablen
```bash
name=wert # Variable setzen (wie set name=wert)
a=1 b=2 # mehrere auf einmal
gruss="Hallo Welt" # Leerzeichen im Wert brauchen Anführungszeichen
heute=$(date +%F) # Ausgabe eines Befehls
PATH=~/bin:$PATH # ~ wird auch nach = und : ersetzt
LANG=C sort datei # nur für diesen einen Befehl
unset name # Variable entfernen
echo $name ${name:-standard}
```
- Eine Zuweisung ist ein Wort `name=wert` am Anfang eines Befehls, bei dem `name` und `=` ohne Anführungszeichen stehen und `name` aus Buchstaben, Ziffern und `_` besteht (nicht mit einer Ziffer beginnend). `"x=1"`, `x\=1` und `echo x=1` sind keine Zuweisungen.
- Steht danach kein Befehl, wird die Variable in der Shell gesetzt. Wie bei `set` und `export` sind in gosh alle Variablen auch für gestartete Programme sichtbar (es gibt keine reinen Shell-Variablen).
- Steht ein Befehl danach (`LANG=C sort datei`), gilt die Zuweisung nur für diesen Befehl, die Shell behält den alten Wert.
- Der Wert wird nicht in Wörter geteilt und nicht als Wildcard oder `{a,b}` expandiert: `x=$(ls)` speichert die ganze Ausgabe, `x=*.go` speichert `*.go`. Variablen, `$(...)` und `~` (am Anfang, nach `=` und nach `:`) werden ersetzt.
- `$?` ist nach einer Zuweisung der Status des letzten `$(...)` darin, sonst 0.
- Variablen in den Argumenten werden ersetzt, bevor die Zuweisung davor gilt: `x=1 echo $x` gibt nicht `1` aus, sondern meldet `x: unbound variable` (wenn `x` vorher nicht gesetzt war).
- In einer Pipeline (`x=1 | cat`) ändert eine Zuweisung die Shell nicht.
### Befehlsersetzung
`$(befehl)` wird durch die Ausgabe von `befehl` ersetzt:
@@ -221,7 +298,7 @@ $ ls -d */ # nur Verzeichnisse
- Versteckte Dateien (mit `.` am Anfang) werden nur gefunden, wenn das Muster selbst mit `.` beginnt: `*` findet `.bashrc` nicht, `.*rc` schon.
- Passt keine Datei, bleibt das Wort unverändert stehen (wie in `sh`), `ls *.xyz` meldet dann also `ls` den Fehler.
- In Anführungszeichen oder mit `\` davor sind die Zeichen normale Zeichen: `echo "*"`, `echo '*.go'`, `echo \*`.
- Werte von Variablen werden nicht als Muster behandelt (anders als in `sh`): Nach `set P=*.go` gibt `echo $P` wörtlich `*.go` aus. So kann ein Variablenwert nie unbemerkt auf viele Dateien passen. `echo $DIR/*.go` funktioniert dagegen, weil das `*` direkt in der Zeile steht.
- Werte von Variablen werden nicht als Muster behandelt (anders als in `sh`): Nach `P='*.go'` gibt `echo $P` wörtlich `*.go` aus. So kann ein Variablenwert nie unbemerkt auf viele Dateien passen. `echo $DIR/*.go` funktioniert dagegen, weil das `*` direkt in der Zeile steht.
- Bei einer Umleitung muss das Muster genau eine Datei treffen (`> *.log`), sonst meldet gosh `ambiguous redirect`.
### Pipes und Umleitungen
+1 -1
View File
@@ -75,7 +75,7 @@ func expandAliasesIn(line string, active map[string]bool) string {
for i := 0; i < len(line); i++ {
ch := line[i]
if quote == 0 && !escaped && commandPos {
if ch == ' ' || ch == '\t' {
if ch == ' ' || ch == '\t' || ch == '\n' {
out.WriteByte(ch)
continue
}
+109
View File
@@ -0,0 +1,109 @@
package main
import (
"os"
"testing"
)
func TestTokenize_Assignments(t *testing.T) {
t.Setenv("HOME", "/home/u")
tests := []struct {
input string
word string
assign bool
}{
{"x=1", "x=1", true},
{"_a1=", "_a1=", true},
{`x="a b"`, "x=a b", true},
{"x=a=b", "x=a=b", true},
{"P=~/bin:~/go/bin", "P=/home/u/bin:/home/u/go/bin", true},
{"x=a~b", "x=a~b", true},
{"x=~", "x=/home/u", true},
// Not an assignment: quoted or escaped name or =, invalid name
{`"x=1"`, "x=1", false},
{`x"=1"`, "x=1", false},
{`x\=1`, "x=1", false},
{"1x=2", "1x=2", false},
{"=x", "=x", false},
{"a-b=1", "a-b=1", false},
}
for _, tc := range tests {
tokens, err := tokenize(tc.input)
if err != nil || len(tokens) != 1 {
t.Errorf("tokenize(%q) = %+v, %v", tc.input, tokens, err)
continue
}
if tokens[0].word != tc.word || tokens[0].assign != tc.assign {
t.Errorf("tokenize(%q) = %q (assign %v), want %q (assign %v)",
tc.input, tokens[0].word, tokens[0].assign, tc.word, tc.assign)
}
}
// The output of $(...) is not split in an assignment
tokens, _ := tokenize("x=$(echo a b) y")
if len(tokens) != 2 || tokens[0].word != "x=a b" || !tokens[0].assign {
t.Errorf("x=$(echo a b): %+v", tokens)
}
}
func TestExec_Assignments(t *testing.T) {
for _, name := range []string{"GOSH_A", "GOSH_B", "GOSH_C"} {
os.Unsetenv(name)
defer os.Unsetenv(name)
}
tests := []struct {
input string
want string
}{
{"GOSH_A=1; echo $GOSH_A > out", "1\n"},
{"GOSH_A=1 GOSH_B=2; echo $GOSH_A$GOSH_B > out", "12\n"},
{`GOSH_A="a b"; echo "$GOSH_A" > out`, "a b\n"},
{"GOSH_A=$(printf 'a b\\nc'); echo \"$GOSH_A\" > out", "a b\nc\n"},
// No wildcard or brace expansion in the value
{"touch x.go; GOSH_A=*.go; echo \"$GOSH_A\" > out", "*.go\n"},
{"GOSH_A={a,b}; echo $GOSH_A > out", "{a,b}\n"},
// ... but as an argument, assignment-like words are normal words
{"echo GOSH_A={a,b} > out", "GOSH_A=a GOSH_A=b\n"},
// Before a command, only in its environment
{"GOSH_C=temp sh -c 'echo $GOSH_C' > out; echo ${GOSH_C:-unset} >> out", "temp\nunset\n"},
{"GOSH_A=old; GOSH_A=new sh -c 'echo $GOSH_A' > out; echo $GOSH_A >> out", "new\nold\n"},
// ... and before a builtin only while it runs
{"GOSH_C=temp pwd > /dev/null; echo ${GOSH_C:-unset} > out", "unset\n"},
// In a pipeline or with a command they do not change the shell
{"GOSH_C=x | cat; echo ${GOSH_C:-unset} > out", "unset\n"},
// Redirections are applied
{"GOSH_A=r > out", ""},
// In loops
{"for i in 1 2 3; do GOSH_A=$i; done; echo $GOSH_A > out", "3\n"},
{"GOSH_B=0; while [ $GOSH_B -lt 3 ]; do echo $GOSH_B >> out; GOSH_B=$(expr $GOSH_B + 1); done", "0\n1\n2\n"},
// Not assignments
{`echo "GOSH_A=1" GOSH_B=2 > out`, "GOSH_A=1 GOSH_B=2\n"},
// unset
{"GOSH_A=1; unset GOSH_A; echo ${GOSH_A:-gone} > out", "gone\n"},
}
for _, tc := range tests {
if got := runIn(t, tc.input); got != tc.want {
t.Errorf("%q: out = %q, want %q", tc.input, got, tc.want)
}
}
// The status of an assignment is the one of its last $(...)
for input, want := range map[string]int{
"GOSH_A=1": 0,
"GOSH_A=$(false)": 1,
"GOSH_A=$(sh -c 'exit 3')": 3,
"GOSH_A=$(false) GOSH_B=$(true)": 0,
} {
runIn(t, input)
if lastStatus != want {
t.Errorf("%q: $? = %d, want %d", input, lastStatus, want)
}
}
if err := execInput("unset 1x"); err == nil || err.Error() != "unset: `1x': not a valid identifier" {
t.Errorf("unset 1x: %v", err)
}
if err := execInput(`"GOSH_A=1"`); err == nil || exitStatus(err) != 127 {
t.Errorf(`"GOSH_A=1" must be a command: %v`, err)
}
}
+19
View File
@@ -22,6 +22,7 @@ func expandBraceWords(input string) (string, error) {
var (
out strings.Builder
afterRedirect bool // the next word is the target of a redirection
commandStart = true
)
for i := 0; i < len(input); {
@@ -33,6 +34,8 @@ func expandBraceWords(input string) (string, error) {
case strings.IndexByte(";&|<>", ch) >= 0:
if ch == '<' || ch == '>' {
afterRedirect = true
} else {
commandStart = true
}
out.WriteByte(ch)
i++
@@ -42,6 +45,16 @@ func expandBraceWords(input string) (string, error) {
return out.String(), nil
default:
end := wordEnd(input, i)
if commandStart && !afterRedirect && isAssignment(input[i:end]) {
// name=value before the command: no brace expansion, like
// in bash
out.WriteString(input[i:end])
i = end
continue
}
if !afterRedirect {
commandStart = false
}
words, err := expandBraces(input[i:end])
if err != nil {
return "", err
@@ -57,6 +70,12 @@ func expandBraceWords(input string) (string, error) {
return out.String(), nil
}
// isAssignment reports whether a word starts with name= (unquoted).
func isAssignment(word string) bool {
name, _, ok := strings.Cut(word, "=")
return ok && isVarName(name)
}
// wordEnd returns the end of the word starting at i: the next unquoted blank
// or operator character.
func wordEnd(s string, i int) int {
+228 -181
View File
@@ -121,6 +121,7 @@ type token struct {
word string // the word, if op == ""
pattern string // the word as glob pattern (quoted characters escaped)
glob bool // the word contains unquoted * ? or [ and is expanded
assign bool // the word is an assignment name=value (unquoted name and =)
fd int // the redirected file descriptor (for redirections)
dup int // the target file descriptor of >& and <&
}
@@ -136,7 +137,8 @@ type token struct {
// - $VAR, ${VAR} and ${VAR:-default} are expanded outside single quotes;
// an unset variable is an error, so that e.g. "rm -rf $DIR/x" never runs
// with an empty $DIR
// - ~ and ~/... are expanded at the start of an unquoted word
// - ~ and ~/... are expanded at the start of an unquoted word, and after
// the = and : of an assignment (PATH=~/bin:~/go/bin)
// - unquoted |, <, >, >>, N<, N>, N>>, N>&M, N<&M, &> and &>> are operators
//
// Expanded values are never split into several words.
@@ -147,6 +149,9 @@ func tokenize(input string) ([]token, error) {
pat strings.Builder // the word as glob pattern, quoted parts escaped
glob bool // the word contains unquoted * ? or [
inWord bool // a word has been started (even if it is still empty)
plain = true // the word so far has no quotes, escapes or expansions
assign bool // the word is an assignment name=value
tildeOK bool // ~ may be expanded here (after = or : of an assignment)
quote byte
escaped bool
rawStart = -1 // start of the current word in input
@@ -154,16 +159,17 @@ func tokenize(input string) ([]token, error) {
endWord := func() {
if inWord {
tokens = append(tokens, token{word: b.String(), pattern: pat.String(), glob: glob})
tokens = append(tokens, token{word: b.String(), pattern: pat.String(), glob: glob, assign: assign})
}
b.Reset()
pat.Reset()
glob = false
glob, assign, plain = false, false, true
inWord = false
rawStart = -1
}
// literal adds quoted, escaped or expanded text: it is never a pattern
literal := func(s string) {
plain = false
b.WriteString(s)
pat.WriteString(escapeGlob(s))
}
@@ -175,6 +181,8 @@ func tokenize(input string) ([]token, error) {
for i := 0; i < len(input); i++ {
ch := input[i]
afterAssignSep := tildeOK
tildeOK = false
switch {
case escaped:
if quote == '"' && !strings.ContainsRune("$\"\\`", rune(ch)) {
@@ -202,7 +210,8 @@ func tokenize(input string) ([]token, error) {
if err != nil {
return nil, err
}
if quote == '"' {
if quote == '"' || assign {
// Quoted or in an assignment, the output is one word
literal(out)
inWord = true
} else {
@@ -241,6 +250,7 @@ func tokenize(input string) ([]token, error) {
}
case ch == '"' || ch == '\'':
startRaw(i)
plain = false
quote = ch
inWord = true
case ch == ' ' || ch == '\t' || ch == '\n':
@@ -296,8 +306,8 @@ func tokenize(input string) ([]token, error) {
i = j - 1
}
tokens = append(tokens, t)
case ch == '~' && !inWord && b.Len() == 0 &&
(i+1 == len(input) || strings.IndexByte("/ \t\n|<>", input[i+1]) >= 0):
case ch == '~' && (!inWord && b.Len() == 0 || afterAssignSep) &&
(i+1 == len(input) || strings.IndexByte("/ \t\n|<>:", input[i+1]) >= 0):
startRaw(i)
home := getHomeDir()
if home == "" {
@@ -307,6 +317,14 @@ func tokenize(input string) ([]token, error) {
inWord = true
default:
startRaw(i)
switch {
case ch == '=' && plain && !assign && isVarName(b.String()):
// name=value; ~ may follow
assign, tildeOK = true, true
case ch == ':' && assign:
// PATH=~/bin:~/go/bin
tildeOK = true
}
b.WriteByte(ch)
switch {
case ch == '*' || ch == '?' || ch == '[':
@@ -742,7 +760,7 @@ type shellCompleter struct {
func newShellCompleter() *shellCompleter {
return &shellCompleter{
commands: []string{"cd", "pwd", "echo", "exit", "quit", "set", "export", "source", "jobs", "fg", "bg", "wait", "alias", "unalias", "help"},
commands: []string{"cd", "pwd", "echo", "exit", "quit", "set", "export", "source", "jobs", "fg", "bg", "wait", "alias", "unalias", "break", "continue", "unset", "help"},
}
}
@@ -756,6 +774,13 @@ func (c *shellCompleter) Do(line []rune, pos int) ([][]rune, int) {
before := strings.TrimSpace(string(line[:start]))
isFirstWord := before == "" || strings.HasSuffix(before, "|") ||
strings.HasSuffix(before, ";") || strings.HasSuffix(before, "&&")
// ... and after if, then, do etc.
if fields := strings.Fields(before); len(fields) > 0 {
switch fields[len(fields)-1] {
case "if", "then", "elif", "else", "do", "while", "until", "!":
isFirstWord = true
}
}
// Plain "~" -> "~/"
if word == "~" {
@@ -1070,28 +1095,56 @@ func sourceFile(cmd, path string) error {
return nil
}
// runScript executes the commands from r line by line. Like sh, it does not
// stop at a failing command and returns the exit status of the last one.
// Empty lines and lines starting with # (e.g. a #!/usr/bin/gosh line) are
// skipped. If name is set, errors are reported with file name and line.
// runScript executes the commands from r. Each command is executed as soon
// as it is complete (a control structure can span several lines). Like sh,
// it does not stop at a failing command and returns the exit status of the
// last one. If name is set, errors are reported with file name and line.
func runScript(r io.Reader, name string) int {
status := 0
for lineNo := 1; ; lineNo++ {
line, err := readLine(r)
if trimmed := strings.TrimSpace(line); trimmed != "" && !strings.HasPrefix(trimmed, "#") {
cmdErr := execInput(line)
status = lastStatus
if name != "" && cmdErr != nil {
cmdErr = fmt.Errorf("%s:%d: %w", sanitize(name), lineNo, cmdErr)
}
printError(cmdErr)
var (
buf strings.Builder
startLine int
)
report := func(err error, line int) {
if name != "" && err != nil {
err = fmt.Errorf("%s:%d: %w", sanitize(name), line, err)
}
if err != nil {
if err != io.EOF {
fmt.Fprintln(os.Stderr, "gosh:", err)
printError(err)
}
for lineNo := 1; ; lineNo++ {
line, readErr := readLine(r)
if readErr == nil || line != "" {
if buf.Len() == 0 {
if strings.TrimSpace(line) == "" {
goto next
}
startLine = lineNo
}
buf.WriteString(line)
buf.WriteByte('\n')
tree, err := parse(buf.String())
if errors.Is(err, errIncomplete) {
goto next // read the rest of the command
}
buf.Reset()
if err != nil {
lastStatus = exitStatus(err)
report(err, startLine)
} else {
report(tree.exec(), startLine)
}
}
next:
if readErr != nil {
if readErr != io.EOF {
fmt.Fprintln(os.Stderr, "gosh:", readErr)
return 1
}
return status
if buf.Len() > 0 {
lastStatus = 2
report(errors.New("syntax error: unexpected end of file"), startLine)
}
return lastStatus
}
}
}
@@ -1129,10 +1182,16 @@ func exitStatus(err error) int {
if errors.As(err, &statusErr) {
return int(statusErr)
}
var synErr syntaxError
if errors.As(err, &synErr) {
var (
synErr syntaxError
lc loopControl
)
if errors.As(err, &synErr) || errors.Is(err, errIncomplete) {
return 2
}
if errors.As(err, &lc) {
return 0
}
var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
if ws, ok := exitErr.Sys().(syscall.WaitStatus); ok && ws.Signaled() {
@@ -1184,23 +1243,35 @@ func interactive() {
}
defer l.Close()
// Main loop
// Main loop. A command that is not complete yet (e.g. if without fi)
// is continued on the next line with the prompt "> ".
var buf strings.Builder
for {
// Report finished and stopped background jobs
reapJobs(true)
// Update prompt dynamically (in case directory changed)
l.SetPrompt(getPrompt())
if buf.Len() == 0 {
// Report finished and stopped background jobs
reapJobs(true)
// Update prompt dynamically (in case directory changed)
l.SetPrompt(getPrompt())
} else {
l.SetPrompt("> ")
}
line, err := l.Readline()
if err != nil {
// Ctrl+C discards the current line
// Ctrl+C discards the current line (and an incomplete command)
if err == readline.ErrInterrupt {
buf.Reset()
continue
}
// Handle EOF (Ctrl+D)
if err == io.EOF {
fmt.Println()
if buf.Len() > 0 {
buf.Reset()
lastStatus = 2
fmt.Fprintln(os.Stderr, "gosh: syntax error: unexpected end of file")
continue
}
if hasStoppedJobs() && !exitWarned {
exitWarned = true
fmt.Fprintln(os.Stderr, "gosh: there are stopped jobs")
@@ -1213,7 +1284,7 @@ func interactive() {
}
// Skip empty input
if strings.TrimSpace(line) == "" {
if buf.Len() == 0 && strings.TrimSpace(line) == "" {
continue
}
@@ -1223,8 +1294,19 @@ func interactive() {
l.SaveHistory(line)
}
// Execute the input
printError(execInput(line))
buf.WriteString(line)
buf.WriteByte('\n')
tree, err := parse(buf.String())
if errors.Is(err, errIncomplete) {
continue
}
buf.Reset()
if err != nil {
lastStatus = exitStatus(err)
printError(err)
continue
}
printError(tree.exec())
}
}
@@ -1256,144 +1338,24 @@ func printError(err error) {
var (
exitErr *exec.ExitError
statusErr statusError
lc loopControl
)
if err == nil || errors.As(err, &exitErr) || errors.As(err, &statusErr) {
if err == nil || errors.As(err, &exitErr) || errors.As(err, &statusErr) || errors.As(err, &lc) {
return
}
fmt.Fprintln(os.Stderr, "gosh:", err)
}
// listItem is a command of a command list together with the operator that
// connects it to the previous command ("" for the first one).
type listItem struct {
op string // "", ";", "&&" or "||"
command string
background bool // the command ends with &
}
// splitList splits input at unquoted ;, &, && and ||. Quotes, escapes, ${...}
// and comments are handled like in splitWords, but nothing is expanded yet:
// that happens per command when it runs, so "cd /tmp && echo $PWD" works.
func splitList(input string) ([]listItem, error) {
var (
items []listItem
op string
start int
quote byte
escaped bool
wordStart = true // the next character starts a word
)
end := len(input)
add := func(i int, next string) error {
command := input[start:i]
if strings.TrimSpace(command) == "" {
// "a;" is fine, but an operator needs a command on both sides
if next != "" || op == "&&" || op == "||" {
if next == "" {
next = op
}
return syntaxError(next)
}
return nil
}
items = append(items, listItem{op: op, command: command})
return nil
}
scan:
for i := 0; i < len(input); i++ {
ch := input[i]
isWordStart := wordStart
wordStart = false
switch {
case escaped:
escaped = false
case quote == '\'':
if ch == '\'' {
quote = 0
}
case ch == '\\':
escaped = true
case isSubstStart(input, i):
// Skip $(...) as a whole, its ; | & belong to the subshell
if j := substEnd(input, i); j >= 0 {
i = j
}
case ch == '$' && i+1 < len(input) && input[i+1] == '{':
// Skip ${...} as a whole, like expandVar does
if j := strings.IndexByte(input[i:], '}'); j >= 0 {
i += j
}
case quote == '"':
if ch == '"' {
quote = 0
}
case ch == '"' || ch == '\'':
quote = ch
case ch == ' ' || ch == '\t' || ch == '\n':
wordStart = true
case ch == '#' && isWordStart:
end = i
break scan
case ch == '&' && !(i+1 < len(input) && strings.IndexByte("&>", input[i+1]) >= 0) &&
!(i > 0 && strings.IndexByte("<>", input[i-1]) >= 0):
// A single & (not &&, &>, >& or <&) runs the command in the
// background
if err := add(i, "&"); err != nil {
return nil, err
}
items[len(items)-1].background = true
op = ";"
start = i + 1
wordStart = true
case ch == ';',
ch == '&' && i+1 < len(input) && input[i+1] == '&',
ch == '|' && i+1 < len(input) && input[i+1] == '|':
next := input[i : i+1]
if ch != ';' {
next = input[i : i+2]
}
if err := add(i, next); err != nil {
return nil, err
}
i += len(next) - 1
op = next
start = i + 1
wordStart = true
}
}
// An unterminated quote or a trailing \ is reported by splitWords
if err := add(end, ""); err != nil {
return nil, err
}
return items, nil
}
// execInput executes a line of input: a list of commands separated by ;, &&
// and ||. It returns the error of the last command that was run; errors of
// earlier commands are printed. $? is updated after each command.
// execInput parses and executes input: a command line or a whole script.
// It returns the error of the last command that was run; errors of earlier
// commands are printed. $? is updated after each command.
func execInput(input string) error {
if interactiveShell {
input = expandAliases(input)
}
items, err := splitList(input)
tree, err := parse(input)
if err != nil {
lastStatus = exitStatus(err)
return err
}
var lastErr error
for _, item := range items {
if (item.op == "&&" && lastStatus != 0) || (item.op == "||" && lastStatus == 0) {
continue
}
printError(lastErr)
lastErr = execCommand(item.command, item.background)
lastStatus = exitStatus(lastErr)
}
return lastErr
return tree.exec()
}
// execCommand executes a pipeline of simple commands with redirections, in
@@ -1447,6 +1409,7 @@ type redirect struct {
// stage is a simple command of a pipeline.
type stage struct {
assigns []string // name=value before the command name
args []string
redirects []redirect
}
@@ -1457,13 +1420,18 @@ func parsePipeline(tokens []token) ([]stage, error) {
stages []stage
cur stage
)
isEmpty := func(s stage) bool { return len(s.args) == 0 && len(s.redirects) == 0 }
isEmpty := func(s stage) bool { return len(s.assigns) == 0 && len(s.args) == 0 && len(s.redirects) == 0 }
for i := 0; i < len(tokens); i++ {
t := tokens[i]
switch t.op {
case "":
cur.args = append(cur.args, expandWord(t)...)
if t.assign && len(cur.args) == 0 {
// The value is not split and not expanded as a pattern
cur.assigns = append(cur.assigns, t.word)
} else {
cur.args = append(cur.args, expandWord(t)...)
}
case "|":
if isEmpty(cur) {
return nil, syntaxError("|")
@@ -1582,6 +1550,20 @@ func startStage(st stage, stdin, stdout *os.File, j *job, inPipeline, background
fds := [3]*os.File{stdin, stdout, os.Stderr}
opened, err := applyRedirects(&fds, st.redirects)
owned = append(owned, opened...)
if err == nil && len(st.args) == 0 && len(st.assigns) > 0 {
// Assignments without a command set variables of the shell (not in
// a pipeline or in the background, where sh runs them in a
// subshell). The status is the one of the last $(...) in them.
if !inPipeline && !background {
for _, a := range st.assigns {
name, value, _ := strings.Cut(a, "=")
os.Setenv(name, value)
}
}
if substStatus != 0 {
err = statusError(substStatus)
}
}
if err != nil || len(st.args) == 0 {
// A command without arguments only creates or truncates the files
closeOwned()
@@ -1608,6 +1590,8 @@ func startStage(st stage, stdin, stdout *os.File, j *job, inPipeline, background
return err
}
if !inPipeline && !background {
// name=value before a builtin applies only while it runs
defer setTemporarily(st.assigns)()
return finishedProc(run())
}
p := &proc{done: make(chan error, 1)}
@@ -1617,6 +1601,11 @@ func startStage(st stage, stdin, stdout *os.File, j *job, inPipeline, background
cmd := exec.Command(st.args[0], st.args[1:]...)
cmd.Stdin, cmd.Stdout, cmd.Stderr = fds[0], fds[1], fds[2]
if len(st.assigns) > 0 {
// name=value before a command is only in its environment (the
// last value of a name wins)
cmd.Env = append(os.Environ(), st.assigns...)
}
if jobControl {
// All processes of the job share a process group: the one of the
// first process. In the foreground, the first process makes its
@@ -1649,6 +1638,31 @@ func startStage(st stage, stdin, stdout *os.File, j *job, inPipeline, background
return &proc{cmd: cmd}
}
// setTemporarily sets the variables of the assignments and returns a
// function that restores the previous values.
func setTemporarily(assigns []string) func() {
type saved struct {
name, value string
ok bool
}
var restore []saved
for _, a := range assigns {
name, value, _ := strings.Cut(a, "=")
old, ok := os.LookupEnv(name)
restore = append(restore, saved{name, old, ok})
os.Setenv(name, value)
}
return func() {
for i := len(restore) - 1; i >= 0; i-- {
if r := restore[i]; r.ok {
os.Setenv(r.name, r.value)
} else {
os.Unsetenv(r.name)
}
}
}
}
// applyRedirects applies the redirections in order to fds (stdin, stdout,
// stderr) and returns the files it opened.
func applyRedirects(fds *[3]*os.File, redirects []redirect) ([]*os.File, error) {
@@ -1701,21 +1715,25 @@ func applyRedirects(fds *[3]*os.File, redirects []redirect) ([]*os.File, error)
// the shell cannot be used in a pipeline, since there they would run
// concurrently (sh runs them in a subshell where they have no effect).
var builtins = map[string]struct{ changesState bool }{
"cd": {changesState: true},
"pwd": {},
"exit": {changesState: true},
"quit": {changesState: true},
"set": {changesState: true},
"export": {changesState: true},
"help": {},
"source": {changesState: true},
".": {changesState: true},
"jobs": {},
"fg": {changesState: true},
"bg": {changesState: true},
"wait": {changesState: true},
"alias": {},
"unalias": {changesState: true},
"cd": {changesState: true},
"pwd": {},
"exit": {changesState: true},
"quit": {changesState: true},
"set": {changesState: true},
"export": {changesState: true},
"help": {},
"source": {changesState: true},
".": {changesState: true},
"jobs": {},
"fg": {changesState: true},
"bg": {changesState: true},
"wait": {changesState: true},
"alias": {},
"unalias": {changesState: true},
":": {},
"unset": {changesState: true},
"break": {changesState: true},
"continue": {changesState: true},
}
// runBuiltin executes a built-in command, writing its output to stdout.
@@ -1795,6 +1813,25 @@ func runBuiltin(args []string, stdout io.Writer) error {
}
os.Setenv(parts[0], parts[1])
return nil
case ":":
// Does nothing, successfully (e.g. while :; do ...)
return nil
case "unset":
// unset NAME ... - remove variables
if len(args) < 2 {
return fmt.Errorf("usage: unset NAME ...")
}
var err error
for _, name := range args[1:] {
if !isVarName(name) {
err = fmt.Errorf("unset: `%s': not a valid identifier", sanitize(name))
continue
}
os.Unsetenv(name)
}
return err
case "break", "continue":
return builtinLoopControl(args)
case "alias":
return builtinAlias(args, stdout)
case "unalias":
@@ -1824,6 +1861,7 @@ Built-in commands:
quit [code] - Quit shell with status code (default: $?)
set VAR=val - Set environment variable
export VAR=val - Export environment variable
unset VAR - Remove a variable
source file - Run the commands of file in this shell (also: . file)
jobs - List background and stopped jobs
fg [%n] - Continue job n (default: current job) in the foreground
@@ -1831,6 +1869,9 @@ Built-in commands:
wait [%n] - Wait for job n or all background jobs
alias [name[=value] ...] - Define or show aliases (alias ll='ls -l')
unalias name - Remove an alias (unalias -a: all aliases)
: - Do nothing (exit status 0)
break [n] - Leave the innermost (or n-th) for/while/until loop
continue [n] - Continue with the next iteration of a loop
help - Show this help
Features:
@@ -1853,6 +1894,9 @@ Features:
GOSH_COLORS='user=1;33:path=36' (keys user, root, path, error; values
are ANSI SGR codes, empty = no color); disabled with $NO_COLOR,
TERM=dumb or when output is not a terminal
- Variables: name=value sets a variable (like set), name=value command
sets it only for that command; the value is not split or expanded as
a pattern
- Environment variable expansion ($VAR, ${VAR}, ${VAR:-default});
using an unset variable is an error, use ${VAR:-} for an empty default
- Command substitution: $(command) is replaced by the output of command
@@ -1860,6 +1904,9 @@ Features:
stays one word
- Command lists: a ; b (run both), a && b (b only if a succeeded),
a || b (b only if a failed); $? is the exit status of the last command
- Control structures: if cmd; then ...; elif cmd; then ...; else ...; fi,
for x in words; do ...; done, while cmd; do ...; done, until ...,
! cmd negates the status; they can span several lines (prompt "> ")
- Pipes (a | b) and redirections: < file, > file, >> file, 2> file,
2>> file, 2>&1, >&2, &> file (stdout and stderr), &>> file
- Job control: command & runs in the background, Ctrl+Z stops the
+154 -32
View File
@@ -842,7 +842,7 @@ func TestShellCompleter_Do(t *testing.T) {
want []string
length int
}{
{"c", []string{"d "}, 1},
{"c", []string{"d ", "ontinue "}, 1},
{"e", []string{"cho ", "xit ", "xport "}, 1},
{"gi", []string{"tty "}, 2},
{" gi", []string{"tty "}, 2},
@@ -1387,48 +1387,170 @@ func TestReadLine_DoesNotReadAhead(t *testing.T) {
}
}
func TestSplitList(t *testing.T) {
// treeString formats a syntax tree for the tests: simple commands in
// brackets, background commands with a trailing &.
func treeString(l *listNode) string {
var parts []string
for _, e := range l.items {
s := pipelineString(e.andOr.first)
for _, r := range e.andOr.rest {
s += " " + r.op + " " + pipelineString(r.pipeline)
}
if e.background {
s += " &"
}
parts = append(parts, s)
}
return strings.Join(parts, "; ")
}
func pipelineString(pl *pipelineNode) string {
s := ""
if pl.negate {
s = "! "
}
switch n := pl.cmd.(type) {
case *simpleNode:
s += "[" + n.text + "]"
case *ifNode:
s += "if"
for i := range n.conds {
if i > 0 {
s += " elif"
}
s += " " + treeString(n.conds[i]) + " then " + treeString(n.bodies[i])
}
if n.elseBody != nil {
s += " else " + treeString(n.elseBody)
}
s += " fi"
case *forNode:
s += "for " + n.name
if n.hasIn {
s += " in (" + n.words + ")"
}
s += " do " + treeString(n.body) + " done"
case *whileNode:
if n.until {
s += "until "
} else {
s += "while "
}
s += treeString(n.cond) + " do " + treeString(n.body) + " done"
}
return s
}
func TestParse_Lists(t *testing.T) {
tests := []struct {
input string
want []listItem
want string
}{
{"", nil},
{" ", nil},
{"# only a comment", nil},
{"ls", []listItem{{"", "ls", false}}},
{"a; b", []listItem{{"", "a", false}, {";", " b", false}}},
{"a;", []listItem{{"", "a", false}}},
{"a && b || c", []listItem{{"", "a ", false}, {"&&", " b ", false}, {"||", " c", false}}},
{`echo "a;b" 'c&&d' e\;f`, []listItem{{"", `echo "a;b" 'c&&d' e\;f`, false}}},
{`echo "x\"; y"`, []listItem{{"", `echo "x\"; y"`, false}}},
{"echo ${X:-a;b}; c", []listItem{{"", "echo ${X:-a;b}", false}, {";", " c", false}}},
{"echo a # b; c", []listItem{{"", "echo a ", false}}},
{"echo a#b; c", []listItem{{"", "echo a#b", false}, {";", " c", false}}},
{"echo a|b", []listItem{{"", "echo a|b", false}}},
{"a & b", []listItem{{"", "a ", true}, {";", " b", false}}},
{"a &", []listItem{{"", "a ", true}}},
{"a&", []listItem{{"", "a", true}}},
{"a 2>&1 &", []listItem{{"", "a 2>&1 ", true}}},
{"a &> f &", []listItem{{"", "a &> f ", true}}},
{"a >&2 && b", []listItem{{"", "a >&2 ", false}, {"&&", " b", false}}},
{`echo "a & b" \&`, []listItem{{"", `echo "a & b" \&`, false}}},
{"", ""},
{" ", ""},
{"# only a comment", ""},
{"ls", "[ls]"},
{"a; b", "[a]; [b]"},
{"a;", "[a]"},
{"a\nb\n", "[a]; [b]"},
{"a && b || c", "[a] && [b] || [c]"},
{"a &&\n b", "[a] && [b]"},
{"a | b|c", "[a | b|c]"},
{"a |\n b", "[a |\n b]"},
{`echo "a;b" 'c&&d' e\;f`, `[echo "a;b" 'c&&d' e\;f]`},
{`echo "x\"; y"`, `[echo "x\"; y"]`},
{"echo ${X:-a;b}; c", "[echo ${X:-a;b}]; [c]"},
{"echo a # b; c", "[echo a]"},
{"echo a#b; c", "[echo a#b]; [c]"},
{"a & b", "[a] &; [b]"},
{"a&", "[a] &"},
{"a 2>&1 &", "[a 2>&1] &"},
{"a &> f &", "[a &> f] &"},
{"a >&2 && b", "[a >&2] && [b]"},
{`echo "a & b" \&`, `[echo "a & b" \&]`},
{"echo $(a; b && c | d) \"$(e || f)\" & g", `[echo $(a; b && c | d) "$(e || f)"] &; [g]`},
{"! false", "! [false]"},
{"echo a \\\n b", "[echo a b]"},
// Reserved words are only special as the first word of a command
{"echo if then fi do done", "[echo if then fi do done]"},
}
for _, tc := range tests {
got, err := splitList(tc.input)
tree, err := parse(tc.input)
if err != nil {
t.Errorf("splitList(%q) returned error: %v", tc.input, err)
t.Errorf("parse(%q) returned error: %v", tc.input, err)
continue
}
if !reflect.DeepEqual(got, tc.want) {
t.Errorf("splitList(%q) = %+v, want %+v", tc.input, got, tc.want)
if got := treeString(tree); got != tc.want {
t.Errorf("parse(%q) = %s, want %s", tc.input, got, tc.want)
}
}
}
func TestSplitList_SyntaxErrors(t *testing.T) {
for _, input := range []string{"; a", "a ;; b", "&& a", "a &&", "a ||", "a && ; b", "a || && b", "& a", "a & && b", "a & & b"} {
if _, err := splitList(input); err == nil || !strings.Contains(err.Error(), "syntax error") {
t.Errorf("splitList(%q) = %v, want syntax error", input, err)
func TestParse_ControlStructures(t *testing.T) {
tests := []struct {
input string
want string
}{
{"if a; then b; fi", "if [a] then [b] fi"},
{"if a\nthen\n b\n c\nfi", "if [a] then [b]; [c] fi"},
{"if a; then b; elif c; then d; else e; fi", "if [a] then [b] elif [c] then [d] else [e] fi"},
{"if a && ! b; then c; fi; echo x", "if [a] && ! [b] then [c] fi; [echo x]"},
{"for x in a b c; do echo $x; done", "for x in (a b c) do [echo $x] done"},
{"for x in *.go {1..3}\ndo\n echo $x\ndone", "for x in (*.go {1..3}) do [echo $x] done"},
{"for x; do echo; done", "for x do [echo] done"},
{"for x in; do echo; done", "for x in () do [echo] done"},
{"while a; do b; done", "while [a] do [b] done"},
{"until a; do b; done", "until [a] do [b] done"},
{"while a; do if b; then break; fi; done", "while [a] do if [b] then [break] fi done"},
{"for i in 1 2; do for j in 3 4; do echo $i$j; done; done", "for i in (1 2) do for j in (3 4) do [echo $i$j] done done"},
{"if a; then if b; then c; fi fi", "if [a] then if [b] then [c] fi fi"},
}
for _, tc := range tests {
tree, err := parse(tc.input)
if err != nil {
t.Errorf("parse(%q) returned error: %v", tc.input, err)
continue
}
if got := treeString(tree); got != tc.want {
t.Errorf("parse(%q) = %s, want %s", tc.input, got, tc.want)
}
}
}
func TestParse_Incomplete(t *testing.T) {
for _, input := range []string{
"if a", "if a; then", "if a; then b", "if a; then b; else", "if a; then b; elif c",
"for x", "for x in a b", "for x in a; do", "for x in a; do b",
"while a", "while a; do b", "until a; do",
"a &&", "a ||", "a |", "echo 'open", `echo "open`, "echo $(a", "echo a \\",
} {
if _, err := parse(input); !errors.Is(err, errIncomplete) {
t.Errorf("parse(%q) = %v, want incomplete", input, err)
}
}
}
func TestParse_SyntaxErrors(t *testing.T) {
for _, input := range []string{
"; a", "a ;; b", "&& a", "a && ; b", "a || && b", "& a", "a & && b", "a & & b",
"fi", "then", "done", "if a; then fi", "if ; then a; fi", "while a; do done",
"if a; then b; done", "for 1x in a; do b; done", "for x in a do b; done; x",
"a | | b", "| a",
} {
_, err := parse(input)
if err == nil || errors.Is(err, errIncomplete) {
t.Errorf("parse(%q) = %v, want syntax error", input, err)
}
}
for input, want := range map[string]string{
"for x in a; do b; done | cat": "control structures in pipelines are not supported",
"echo | while a; do b; done": "control structures in pipelines are not supported",
"if a; then b; fi > out": "redirections of control structures are not supported (near `>')",
"for x in a; do b; done &": "only simple commands and pipelines can run in the background",
"a && b &": "only simple commands and pipelines can run in the background",
} {
if _, err := parse(input); err == nil || err.Error() != want {
t.Errorf("parse(%q) = %v, want %q", input, err, want)
}
}
}
@@ -1552,7 +1674,7 @@ func TestTokenize_Operators(t *testing.T) {
}
// Patterns are tested in TestTokenize_Patterns
for i := range got {
got[i].pattern, got[i].glob = "", false
got[i].pattern, got[i].glob, got[i].assign = "", false, false
}
if !reflect.DeepEqual(got, tc.want) {
t.Errorf("tokenize(%q) = %+v, want %+v", tc.input, got, tc.want)
+841
View File
@@ -0,0 +1,841 @@
package main
// Parser and executor for command lines and scripts: lists (; & && || and
// newlines), pipelines, ! and the control structures if, for, while and
// until. Simple commands (with their pipes and redirections) are kept as
// text; they are expanded and run by execCommand only when they are
// executed, so that e.g. the variable of a for loop is expanded anew in
// every iteration.
import (
"errors"
"fmt"
"os"
"strconv"
"strings"
"syscall"
)
// incompleteError is returned by parse if the input ends inside a
// construct (if without fi, an open quote, a trailing && ...): an
// interactive shell then reads another line.
type incompleteError struct{}
func (incompleteError) Error() string { return "syntax error: unexpected end of input" }
var errIncomplete error = incompleteError{}
// loopControl is returned by break and continue and ends the given number of
// enclosing loops (break) or iterations (continue).
type loopControl struct {
isBreak bool
n int
}
func (c loopControl) Error() string {
if c.isBreak {
return "break"
}
return "continue"
}
// loopDepth is the number of loops that are currently running.
var loopDepth int
// aliasExpansion is false while the result of an alias expansion runs, so
// that it is not expanded again.
var aliasExpansion = true
// reserved words that end a list; they are only recognized as the first word
// of a command.
var terminators = map[string]bool{"then": true, "elif": true, "else": true, "fi": true, "do": true, "done": true}
// ---------------------------------------------------------------------------
// Lexer
type tokenKind int
const (
tWord tokenKind = iota
tOp // ; & && || | or newline
)
type rawToken struct {
kind tokenKind
text string
start, end int
}
// joinContinuationLines removes backslash-newline (outside single quotes),
// which continues a line. continued reports whether src ends with such a
// continuation (so that the command goes on in the next line).
func joinContinuationLines(src string) (joined string, continued bool) {
if !strings.Contains(src, "\\\n") {
return src, false
}
var b strings.Builder
var quote byte
for i := 0; i < len(src); i++ {
ch := src[i]
switch {
case quote == '\'':
if ch == '\'' {
quote = 0
}
case ch == '\\' && i+1 < len(src):
if src[i+1] == '\n' {
i++
continued = i == len(src)-1
continue
}
continued = false
b.WriteByte(ch)
i++
ch = src[i]
case ch == '"':
if quote == '"' {
quote = 0
} else {
quote = '"'
}
case ch == '\'' && quote == 0:
quote = '\''
}
if ch != '\n' && ch != ' ' && ch != '\t' {
continued = false
}
b.WriteByte(ch)
}
return b.String(), continued
}
// lex splits src into words and list operators. Words keep their quotes,
// escapes, $(...) and redirections; they are only split at blanks and list
// operators.
func lex(src string) ([]rawToken, error) {
var toks []rawToken
op := func(text string, i int) {
toks = append(toks, rawToken{tOp, text, i, i + len(text)})
}
for i := 0; i < len(src); {
ch := src[i]
next := byte(0)
if i+1 < len(src) {
next = src[i+1]
}
switch {
case ch == ' ' || ch == '\t':
i++
case ch == '\n' || ch == ';':
op(string(ch), i)
i++
case ch == '#':
// Comment until the end of the line
for i < len(src) && src[i] != '\n' {
i++
}
case ch == '|' && next == '|', ch == '&' && next == '&':
op(src[i:i+2], i)
i += 2
case ch == '|', ch == '&' && next != '>':
op(string(ch), i)
i++
default:
end, err := lexWord(src, i)
if err != nil {
return nil, err
}
toks = append(toks, rawToken{tWord, src[i:end], i, end})
i = end
}
}
return toks, nil
}
// lexWord returns the end of the word that starts at src[i].
func lexWord(src string, i int) (int, error) {
var quote byte
for j := i; j < len(src); j++ {
ch := src[j]
switch {
case quote == '\'':
if ch == '\'' {
quote = 0
}
case ch == '\\':
if j+1 == len(src) {
return 0, errIncomplete
}
j++
case isSubstStart(src, j):
end := substEnd(src, j)
if end < 0 {
return 0, errIncomplete
}
j = end
case ch == '$' && j+1 < len(src) && src[j+1] == '{':
if k := strings.IndexByte(src[j:], '}'); k >= 0 {
j += k
}
case quote == '"':
if ch == '"' {
quote = 0
}
case ch == '"' || ch == '\'':
quote = ch
case ch == ' ' || ch == '\t' || ch == '\n' || ch == ';' || ch == '|':
return j, nil
case ch == '&':
// &>, >& and <& are redirections, not the & operator
if !(j+1 < len(src) && src[j+1] == '>') && !(j > i && (src[j-1] == '>' || src[j-1] == '<')) {
return j, nil
}
}
}
if quote != 0 {
return 0, errIncomplete
}
return len(src), nil
}
// ---------------------------------------------------------------------------
// Syntax tree
// node is a command that can be executed: it returns the error of the last
// command it ran (so that exitStatus gives its status).
type node interface {
exec() error
}
// listNode is a list of and-or lists separated by ;, & or newlines.
type listNode struct {
items []listEntry
}
type listEntry struct {
andOr *andOrNode
background bool
}
// andOrNode is a pipeline followed by && and || pipelines.
type andOrNode struct {
first *pipelineNode
rest []andOrPart
}
type andOrPart struct {
op string // && or ||
pipeline *pipelineNode
}
// pipelineNode is a simple command (whose text may contain a pipeline) or a
// control structure, optionally negated with !.
type pipelineNode struct {
negate bool
cmd node
}
// simpleNode is the text of a pipeline of simple commands.
type simpleNode struct {
text string
}
type ifNode struct {
conds []*listNode
bodies []*listNode
elseBody *listNode
}
type forNode struct {
name string
words string // the words after "in", expanded when the loop starts
hasIn bool
body *listNode
}
type whileNode struct {
until bool
cond *listNode
body *listNode
}
// singleSimple returns the simple command if the list is just that.
func (l *listNode) singleSimple() *simpleNode {
if len(l.items) != 1 || l.items[0].background {
return nil
}
ao := l.items[0].andOr
if len(ao.rest) > 0 || ao.first.negate {
return nil
}
s, _ := ao.first.cmd.(*simpleNode)
return s
}
// ---------------------------------------------------------------------------
// Parser
type parser struct {
src string
toks []rawToken
pos int
}
// parse parses a command line or script. It returns errIncomplete if the
// input ends inside a construct.
func parse(src string) (*listNode, error) {
src, continued := joinContinuationLines(src)
if continued {
return nil, errIncomplete
}
toks, err := lex(src)
if err != nil {
return nil, err
}
p := &parser{src: src, toks: toks}
list, err := p.parseList(nil)
if err != nil {
return nil, err
}
if t := p.peek(); t != nil {
return nil, p.unexpected(t)
}
return list, nil
}
func (p *parser) peek() *rawToken {
if p.pos < len(p.toks) {
return &p.toks[p.pos]
}
return nil
}
func (p *parser) peekWord(words ...string) bool {
t := p.peek()
if t == nil || t.kind != tWord {
return false
}
for _, w := range words {
if t.text == w {
return true
}
}
return false
}
func (p *parser) skipNewlines() {
for t := p.peek(); t != nil && t.kind == tOp && t.text == "\n"; t = p.peek() {
p.pos++
}
}
func (p *parser) unexpected(t *rawToken) error {
if t.text == "\n" {
return syntaxError("newline")
}
return syntaxError(t.text)
}
// expect consumes the reserved word w.
func (p *parser) expect(w string) error {
t := p.peek()
if t == nil {
return errIncomplete
}
if t.kind != tWord || t.text != w {
return p.unexpected(t)
}
p.pos++
return nil
}
// parseList parses and-or lists until the end of the input or one of the
// reserved words in terms at the start of a command.
func (p *parser) parseList(terms map[string]bool) (*listNode, error) {
list := &listNode{}
for {
p.skipNewlines()
t := p.peek()
if t == nil || t.kind == tWord && terms[t.text] {
return list, nil
}
ao, err := p.parseAndOr()
if err != nil {
return nil, err
}
entry := listEntry{andOr: ao}
t = p.peek()
switch {
case t == nil:
case t.kind == tOp && (t.text == ";" || t.text == "&" || t.text == "\n"):
p.pos++
entry.background = t.text == "&"
case t.kind == tWord && terms[t.text]:
// e.g. "fi" right after the end of a nested control structure
default:
return nil, p.unexpected(t)
}
if entry.background {
if _, ok := ao.first.cmd.(*simpleNode); !ok || len(ao.rest) > 0 || ao.first.negate {
return nil, errors.New("only simple commands and pipelines can run in the background")
}
}
list.items = append(list.items, entry)
}
}
// nonEmpty returns an error if a list that must contain commands is empty.
func (p *parser) nonEmpty(list *listNode) error {
if len(list.items) > 0 {
return nil
}
if t := p.peek(); t != nil {
return p.unexpected(t)
}
return errIncomplete
}
func (p *parser) parseAndOr() (*andOrNode, error) {
first, err := p.parsePipeline()
if err != nil {
return nil, err
}
ao := &andOrNode{first: first}
for {
t := p.peek()
if t == nil || t.kind != tOp || (t.text != "&&" && t.text != "||") {
return ao, nil
}
p.pos++
p.skipNewlines()
if p.peek() == nil {
return nil, errIncomplete
}
next, err := p.parsePipeline()
if err != nil {
return nil, err
}
ao.rest = append(ao.rest, andOrPart{op: t.text, pipeline: next})
}
}
func (p *parser) parsePipeline() (*pipelineNode, error) {
pl := &pipelineNode{}
if p.peekWord("!") {
pl.negate = true
p.pos++
}
t := p.peek()
if t == nil {
return nil, errIncomplete
}
if t.kind == tOp || terminators[t.text] {
return nil, p.unexpected(t)
}
var err error
compound := true
switch t.text {
case "if":
pl.cmd, err = p.parseIf()
case "for":
pl.cmd, err = p.parseFor()
case "while", "until":
pl.cmd, err = p.parseWhile()
default:
compound = false
pl.cmd, err = p.parseSimple()
}
if err != nil {
return nil, err
}
if compound {
if t := p.peek(); t != nil {
switch {
case t.kind == tOp && t.text == "|":
return nil, errors.New("control structures in pipelines are not supported")
case t.kind == tWord && !terminators[t.text]:
return nil, fmt.Errorf("redirections of control structures are not supported (near `%s')", t.text)
}
}
}
return pl, nil
}
// parseSimple parses a pipeline of simple commands and returns its text.
func (p *parser) parseSimple() (*simpleNode, error) {
start := p.toks[p.pos].start
end := start
for t := p.peek(); t != nil; t = p.peek() {
if t.kind == tWord {
end = t.end
p.pos++
continue
}
if t.text != "|" {
break
}
p.pos++
end = t.end
p.skipNewlines()
next := p.peek()
switch {
case next == nil:
return nil, errIncomplete
case next.kind == tOp || terminators[next.text]:
return nil, p.unexpected(next)
case next.text == "if" || next.text == "for" || next.text == "while" || next.text == "until":
return nil, errors.New("control structures in pipelines are not supported")
}
}
return &simpleNode{text: p.src[start:end]}, nil
}
func (p *parser) parseIf() (*ifNode, error) {
p.pos++ // if
n := &ifNode{}
for {
cond, err := p.parseList(map[string]bool{"then": true})
if err == nil {
err = p.nonEmpty(cond)
}
if err == nil {
err = p.expect("then")
}
if err != nil {
return nil, err
}
body, err := p.parseList(map[string]bool{"elif": true, "else": true, "fi": true})
if err == nil {
err = p.nonEmpty(body)
}
if err != nil {
return nil, err
}
n.conds = append(n.conds, cond)
n.bodies = append(n.bodies, body)
t := p.peek()
if t == nil {
return nil, errIncomplete
}
p.pos++
switch t.text {
case "elif":
continue
case "else":
body, err := p.parseList(map[string]bool{"fi": true})
if err == nil {
err = p.nonEmpty(body)
}
if err == nil {
err = p.expect("fi")
}
if err != nil {
return nil, err
}
n.elseBody = body
}
return n, nil // fi
}
}
func (p *parser) parseFor() (*forNode, error) {
p.pos++ // for
t := p.peek()
if t == nil {
return nil, errIncomplete
}
if t.kind != tWord || !isVarName(t.text) {
return nil, fmt.Errorf("for: `%s': not a valid identifier", sanitize(t.text))
}
n := &forNode{name: t.text}
p.pos++
p.skipNewlines()
if p.peekWord("in") {
p.pos++
n.hasIn = true
var words []string
for t := p.peek(); t != nil && t.kind == tWord; t = p.peek() {
words = append(words, t.text)
p.pos++
}
n.words = strings.Join(words, " ")
}
t = p.peek()
switch {
case t == nil:
return nil, errIncomplete
case t.kind == tOp && (t.text == ";" || t.text == "\n"):
p.pos++
case !n.hasIn && t.kind == tWord && t.text == "do":
default:
return nil, p.unexpected(t)
}
p.skipNewlines()
body, err := p.parseLoopBody()
if err != nil {
return nil, err
}
n.body = body
return n, nil
}
func (p *parser) parseWhile() (*whileNode, error) {
n := &whileNode{until: p.peek().text == "until"}
p.pos++
cond, err := p.parseList(map[string]bool{"do": true})
if err == nil {
err = p.nonEmpty(cond)
}
if err != nil {
return nil, err
}
n.cond = cond
body, err := p.parseLoopBody()
if err != nil {
return nil, err
}
n.body = body
return n, nil
}
// parseLoopBody parses "do list done".
func (p *parser) parseLoopBody() (*listNode, error) {
if err := p.expect("do"); err != nil {
return nil, err
}
body, err := p.parseList(map[string]bool{"done": true})
if err == nil {
err = p.nonEmpty(body)
}
if err == nil {
err = p.expect("done")
}
return body, err
}
// ---------------------------------------------------------------------------
// Execution
// isAbort reports whether err stops the execution of the enclosing lists:
// break, continue, or a command that was interrupted with Ctrl+C (like in
// bash, Ctrl+C aborts the whole command line, e.g. a loop).
func isAbort(err error) bool {
var lc loopControl
return errors.As(err, &lc) || exitStatus(err) == 128+int(syscall.SIGINT)
}
// silent returns an error with only the exit status of err, for errors that
// were already reported.
func silent(err error) error {
if err == nil {
return nil
}
return statusError(exitStatus(err))
}
func (l *listNode) exec() error {
var err error
for _, entry := range l.items {
printError(err)
if entry.background {
err = entry.andOr.first.cmd.(*simpleNode).run(true)
} else {
err = entry.andOr.exec()
}
lastStatus = exitStatus(err)
if isAbort(err) {
return err
}
}
return err
}
func (a *andOrNode) exec() error {
err := a.first.exec()
for _, part := range a.rest {
lastStatus = exitStatus(err)
if isAbort(err) {
return err
}
if (part.op == "&&") != (lastStatus == 0) {
continue
}
printError(err)
err = part.pipeline.exec()
}
return err
}
func (pl *pipelineNode) exec() error {
err := pl.cmd.exec()
if !pl.negate || isAbort(err) {
return err
}
printError(err)
if exitStatus(err) == 0 {
return statusError(1)
}
return nil
}
func (s *simpleNode) exec() error {
return s.run(false)
}
// run executes the simple command, in the background if background is set.
// Aliases are expanded here (in interactive shells), so that they work at
// the start of every command, also inside control structures.
func (s *simpleNode) run(background bool) error {
text := s.text
if interactiveShell && aliasExpansion {
if expanded := expandAliases(text); expanded != text {
tree, err := parse(expanded)
if err != nil {
return err
}
if simple := tree.singleSimple(); simple != nil {
text = simple.text
} else {
if background {
return errors.New("an alias with a command list cannot run in the background")
}
aliasExpansion = false
defer func() { aliasExpansion = true }()
return tree.exec()
}
}
}
return execCommand(text, background)
}
func (n *ifNode) exec() error {
for i, cond := range n.conds {
err := cond.exec()
if isAbort(err) {
return err
}
printError(err)
if exitStatus(err) == 0 {
return n.bodies[i].exec()
}
}
if n.elseBody != nil {
return n.elseBody.exec()
}
lastStatus = 0
return nil
}
// runLoopBody runs one iteration of a loop. It returns whether the loop
// must stop, and the error to return from the loop.
func runLoopBody(body *listNode, result *error) (stop bool, err error) {
err = body.exec()
var lc loopControl
if errors.As(err, &lc) {
*result = nil
if lc.n > 1 {
lc.n--
return true, lc // ends an outer loop too
}
return lc.isBreak, nil
}
if isAbort(err) {
return true, err
}
printError(err)
*result = silent(err)
return false, nil
}
func (n *whileNode) exec() error {
loopDepth++
defer func() { loopDepth-- }()
var result error
for {
err := n.cond.exec()
if isAbort(err) {
return err
}
printError(err)
if (exitStatus(err) == 0) == n.until {
break
}
if stop, err := runLoopBody(n.body, &result); stop {
if err != nil {
return err
}
break
}
}
lastStatus = exitStatus(result)
return result
}
func (n *forNode) exec() error {
var words []string
if n.hasIn {
var err error
if words, err = expandWordList(n.words); err != nil {
return err
}
}
loopDepth++
defer func() { loopDepth-- }()
var result error
for _, word := range words {
os.Setenv(n.name, word)
if stop, err := runLoopBody(n.body, &result); stop {
if err != nil {
return err
}
break
}
}
lastStatus = exitStatus(result)
return result
}
// expandWordList expands the words of a for loop like the arguments of a
// command (braces, ~, variables, $(...), wildcards).
func expandWordList(text string) ([]string, error) {
text, err := expandBraceWords(text)
if err != nil {
return nil, err
}
tokens, err := tokenize(text)
if err != nil {
return nil, fmt.Errorf("invalid input: %w", err)
}
var words []string
for _, t := range tokens {
if t.op != "" {
return nil, syntaxError(t.op)
}
words = append(words, expandWord(t)...)
}
return words, nil
}
// builtinLoopControl implements break [n] and continue [n].
func builtinLoopControl(args []string) error {
n := 1
if len(args) > 1 {
var err error
if n, err = strconv.Atoi(args[1]); err != nil || n < 1 {
return fmt.Errorf("%s: %s: loop count out of range", args[0], sanitize(args[1]))
}
}
if loopDepth == 0 {
return fmt.Errorf("%s: only meaningful in a loop", args[0])
}
return loopControl{isBreak: args[0] == "break", n: min(n, loopDepth)}
}
+164
View File
@@ -0,0 +1,164 @@
package main
import (
"os"
"strings"
"testing"
)
// runIn runs input in a temporary directory and returns the content of the
// file "out" there.
func runIn(t *testing.T, input string) string {
t.Helper()
origDir, _ := os.Getwd()
defer os.Chdir(origDir)
os.Chdir(t.TempDir())
execInput(input)
data, _ := os.ReadFile("out")
return string(data)
}
func TestExec_If(t *testing.T) {
tests := []struct {
input string
want string
}{
{"if true; then echo yes >> out; fi", "yes\n"},
{"if false; then echo yes >> out; fi", ""},
{"if false; then echo a >> out; else echo b >> out; fi", "b\n"},
{"if false; then echo a >> out; elif true; then echo b >> out; else echo c >> out; fi", "b\n"},
{"if false; then echo a >> out; elif false; then echo b >> out; else echo c >> out; fi", "c\n"},
{"if ! false; then echo negated >> out; fi", "negated\n"},
{"if true && false; then echo a >> out; else echo b >> out; fi", "b\n"},
{"if false || true; then echo a >> out; fi", "a\n"},
{"if echo cond >> out; then echo body >> out; fi", "cond\nbody\n"},
{"if true\nthen\n echo multi >> out\n echo line >> out\nfi", "multi\nline\n"},
{"if true; then if false; then echo a >> out; else echo nested >> out; fi; fi", "nested\n"},
{"if [ -d / ]; then echo dir >> out; fi", "dir\n"},
}
for _, tc := range tests {
if got := runIn(t, tc.input); got != tc.want {
t.Errorf("%q: out = %q, want %q", tc.input, got, tc.want)
}
}
}
func TestExec_Loops(t *testing.T) {
defer os.Unsetenv("GOSH_I")
tests := []struct {
input string
want string
}{
{"for x in a b c; do echo $x >> out; done", "a\nb\nc\n"},
{"for x in {1..3}; do echo $x >> out; done", "1\n2\n3\n"},
{"for x in $(echo a b); do echo $x >> out; done", "a\nb\n"},
{"for x in \"a b\" c; do echo $x >> out; done", "a b\nc\n"},
{"touch f1 f2; for f in f*; do echo $f >> out; done", "f1\nf2\n"},
{"for x in; do echo never >> out; done", ""},
{"for x; do echo never >> out; done", ""},
{"set GOSH_I=0; while [ $GOSH_I -lt 3 ]; do echo $GOSH_I >> out; set GOSH_I=$(expr $GOSH_I + 1); done", "0\n1\n2\n"},
{"set GOSH_I=3; until [ $GOSH_I -eq 0 ]; do echo $GOSH_I >> out; set GOSH_I=$(expr $GOSH_I - 1); done", "3\n2\n1\n"},
{"while false; do echo never >> out; done", ""},
// The loop variable keeps the last value
{"for GOSH_I in a b; do :; done; echo $GOSH_I > out", "b\n"},
// break and continue
{"for x in 1 2 3 4; do if [ $x = 3 ]; then break; fi; echo $x >> out; done", "1\n2\n"},
{"for x in 1 2 3; do if [ $x = 2 ]; then continue; fi; echo $x >> out; done", "1\n3\n"},
{"for a in 1 2; do for b in x y; do if [ $b = y ]; then continue 2; fi; echo $a$b >> out; done; done", "1x\n2x\n"},
{"for a in 1 2; do for b in x y; do echo $a$b >> out; break 2; done; done", "1x\n"},
{"for a in 1 2; do for b in x y; do echo $a$b >> out; break 5; done; done", "1x\n"},
{"while true; do echo once >> out; break; done", "once\n"},
{"for x in 1 2; do echo $x >> out; break; echo never >> out; done; echo after >> out", "1\nafter\n"},
}
for _, tc := range tests {
if got := runIn(t, tc.input); got != tc.want {
t.Errorf("%q: out = %q, want %q", tc.input, got, tc.want)
}
}
if loopDepth != 0 {
t.Errorf("loopDepth = %d after the loops", loopDepth)
}
}
func TestExec_Status(t *testing.T) {
tests := []struct {
input string
status int
}{
{"if false; then :; fi", 0},
{"if true; then false; fi", 1},
{"if false; then :; else sh -c 'exit 3'; fi", 3},
{"for x in a; do false; done", 1},
{"for x in a; do false; done; true", 0},
{"for x in; do false; done", 0},
{"while false; do :; done", 0},
{"for x in a b; do sh -c 'exit 4'; break; done", 0},
{"! true", 1},
{"! false", 0},
{"! gosh-no-such-command", 0},
{":", 0},
{"break", 1},
{"continue", 1},
{"for x in a; do break x; done", 1},
{"if a; then", 2},
{"fi", 2},
}
for _, tc := range tests {
runIn(t, tc.input)
if lastStatus != tc.status {
t.Errorf("%q: $? = %d, want %d", tc.input, lastStatus, tc.status)
}
}
if err := execInput("break"); err == nil || err.Error() != "break: only meaningful in a loop" {
t.Errorf("break outside a loop: %v", err)
}
}
func TestRunScript_MultiLine(t *testing.T) {
origDir, _ := os.Getwd()
defer os.Chdir(origDir)
os.Chdir(t.TempDir())
script := `# a script with control structures
for x in a b
do
if [ $x = a ]; then
echo first >> out
else
echo second >> out
fi
done
echo continued \
line >> out
`
if status := runScript(strings.NewReader(script), ""); status != 0 {
t.Errorf("status = %d", status)
}
if got, _ := os.ReadFile("out"); string(got) != "first\nsecond\ncontinued line\n" {
t.Errorf("out = %q", got)
}
// A script that ends inside a control structure is a syntax error
if status := runScript(strings.NewReader("echo ok >> out2\nif true; then\n echo never >> out2\n"), ""); status != 2 {
t.Errorf("status = %d for an unterminated if, want 2", status)
}
if got, _ := os.ReadFile("out2"); string(got) != "ok\n" {
t.Errorf("out2 = %q", got)
}
}
func TestExec_AliasesInControlStructures(t *testing.T) {
withAliases(t, map[string]string{"say": "echo said", "both": "echo one >> out; echo two"})
defer func(saved bool) { interactiveShell = saved }(interactiveShell)
interactiveShell = true
if got := runIn(t, "for x in a b; do say $x >> out; done"); got != "said a\nsaid b\n" {
t.Errorf("alias in for: out = %q", got)
}
if got := runIn(t, "if true; then both >> out; fi"); got != "one\ntwo\n" {
t.Errorf("alias with a list in if: out = %q", got)
}
if got := runIn(t, "if say x > out; then :; fi"); got != "said x\n" {
t.Errorf("alias in condition: out = %q", got)
}
}
+1 -13
View File
@@ -2,7 +2,6 @@ package main
import (
"os"
"reflect"
"testing"
)
@@ -29,17 +28,6 @@ func TestSubstEnd(t *testing.T) {
}
}
func TestSplitList_SkipsSubstitution(t *testing.T) {
got, err := splitList(`echo $(a; b && c | d) "$(e || f)" & g`)
if err != nil {
t.Fatal(err)
}
want := []listItem{{"", `echo $(a; b && c | d) "$(e || f)" `, true}, {";", " g", false}}
if !reflect.DeepEqual(got, want) {
t.Errorf("splitList = %+v, want %+v", got, want)
}
}
func TestExpandAliasesAndBraces_SkipSubstitution(t *testing.T) {
withAliases(t, map[string]string{"ll": "ls -l"})
if got := expandAliases("echo $(ll; ll) && ll"); got != "echo $(ll; ll) && ls -l" {
@@ -106,7 +94,7 @@ func TestExecInput_CommandSubstitution(t *testing.T) {
}
for input, wantErr := range map[string]string{
"echo $(echo a": "invalid input: missing ) in command substitution",
"echo $(echo a": "syntax error: unexpected end of input",
"echo $((1 + 2))": "invalid input: arithmetic expansion $((...)) is not supported",
} {
if err := execInput(input); err == nil || err.Error() != wantErr {