add pipes and redirections
- tokenizer recognizes unquoted |, <, >, >>, N>, N>>, N>&M, N<&M, &> and &>>; only a word of unquoted digits is a file descriptor - pipelines run concurrently, the status is the one of the last command, errors of earlier stages are printed - redirections are applied left to right (fds 0-2), a redirection without a command creates/truncates the file - pwd and help can be redirected and piped; cd, set, export, exit and quit are rejected in pipelines - syntax errors have exit status 2 - completion: commands after |, ;, && and ||, files after redirections; operator characters are escaped in completed names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
f5c14dff90
commit
c46903e3d2
3 files changed
+570
-43
No files matched your search
@@ -31,6 +31,8 @@ Features:
|
|||||||
using an unset variable is an error, use ${VAR:-} for an empty default
|
using an unset variable is an error, use ${VAR:-} for an empty default
|
||||||
- Command lists: a ; b (run both), a && b (b only if a succeeded),
|
- Command lists: a ; b (run both), a && b (b only if a succeeded),
|
||||||
a || b (b only if a failed); $? is the exit status of the last command
|
a || b (b only if a failed); $? is the exit status of the last command
|
||||||
|
- Pipes (a | b) and redirections: < file, > file, >> file, 2> file,
|
||||||
|
2>> file, 2>&1, >&2, &> file (stdout and stderr), &>> file
|
||||||
- Non-interactive use: gosh -c 'command', gosh script, command | gosh;
|
- Non-interactive use: gosh -c 'command', gosh script, command | gosh;
|
||||||
the exit status is the one of the last command
|
the exit status is the one of the last command
|
||||||
|
|
||||||
@@ -55,7 +57,7 @@ gosh -version
|
|||||||
|
|
||||||
Ohne Terminal bricht gosh bei einem fehlgeschlagenen Befehl nicht ab und beendet sich mit dem Exit-Status des letzten Befehls: `127`, wenn ein Befehl nicht gefunden wurde, `126`, wenn er nicht ausführbar ist, `128+n`, wenn er durch Signal `n` beendet wurde. Leere Zeilen und Zeilen, die mit `#` beginnen, werden übersprungen, Skripte können also mit `#!/usr/bin/gosh` beginnen.
|
Ohne Terminal bricht gosh bei einem fehlgeschlagenen Befehl nicht ab und beendet sich mit dem Exit-Status des letzten Befehls: `127`, wenn ein Befehl nicht gefunden wurde, `126`, wenn er nicht ausführbar ist, `128+n`, wenn er durch Signal `n` beendet wurde. Leere Zeilen und Zeilen, die mit `#` beginnen, werden übersprungen, Skripte können also mit `#!/usr/bin/gosh` beginnen.
|
||||||
|
|
||||||
Damit funktionieren auch `ssh host befehl`, `scp` und `rsync`, wenn gosh als Login-Shell gesetzt ist. Befehle lassen sich mit `;`, `&&` und `||` verketten (z. B. `ssh host 'cd /srv && git pull'`), Pipes und Umleitungen kennt gosh aber noch nicht.
|
Damit funktionieren auch `ssh host befehl`, `scp` und `rsync`, wenn gosh als Login-Shell gesetzt ist. Befehle lassen sich mit `;`, `&&` und `||` verketten und mit Pipes verbinden, z. B. `ssh host 'cd /srv && git log | head'`.
|
||||||
|
|
||||||
### Befehlslisten und Exit-Status
|
### Befehlslisten und Exit-Status
|
||||||
|
|
||||||
@@ -81,6 +83,33 @@ Status 127
|
|||||||
|
|
||||||
Schlägt ein Programm fehl, gibt gosh selbst keine Meldung mehr aus (das Programm meldet den Fehler selbst), nur bei Fehlern von gosh, z. B. wenn ein Befehl nicht gefunden wird.
|
Schlägt ein Programm fehl, gibt gosh selbst keine Meldung mehr aus (das Programm meldet den Fehler selbst), nur bei Fehlern von gosh, z. B. wenn ein Befehl nicht gefunden wird.
|
||||||
|
|
||||||
|
### Pipes und Umleitungen
|
||||||
|
|
||||||
|
```
|
||||||
|
$ ls | sort -r | head -3
|
||||||
|
$ make > build.log 2>&1
|
||||||
|
$ grep -c error < build.log
|
||||||
|
```
|
||||||
|
|
||||||
|
| Syntax | Bedeutung |
|
||||||
|
|-----------------|------------------------------------------------------------|
|
||||||
|
| `a \| b` | stdout von `a` wird stdin von `b` |
|
||||||
|
| `< datei` | stdin aus `datei` lesen |
|
||||||
|
| `> datei` | stdout in `datei` schreiben (wird überschrieben) |
|
||||||
|
| `>> datei` | stdout an `datei` anhängen |
|
||||||
|
| `2> datei` | stderr in `datei` schreiben (`2>>` hängt an) |
|
||||||
|
| `2>&1` | stderr dorthin, wohin stdout gerade geht |
|
||||||
|
| `>&2` | stdout dorthin, wohin stderr gerade geht |
|
||||||
|
| `&> datei` | stdout und stderr in `datei` (`&>>` hängt an) |
|
||||||
|
| `> datei` | ohne Befehl: legt `datei` an bzw. leert sie |
|
||||||
|
|
||||||
|
- Umleitungen werden von links nach rechts ausgewertet: `befehl > log 2>&1` schreibt beides in `log`, `befehl 2>&1 > log` schickt stderr dorthin, wo stdout vorher hinging (z. B. in eine Pipe).
|
||||||
|
- Unterstützt werden die Dateideskriptoren 0, 1 und 2. Vor `>` und `<` zählt nur eine Zahl ohne Anführungszeichen als Deskriptor (`2>f`, aber nicht `x2>f` oder `"2">f`).
|
||||||
|
- Operatoren in Anführungszeichen oder mit `\` davor sind normale Zeichen: `echo "a|b" 'c>d' e\>f`.
|
||||||
|
- Der Exit-Status einer Pipeline ist der des letzten Befehls. Beendet sich ein Befehl am Ende (z. B. `head`), bekommt der schreibende Befehl davor `SIGPIPE`, `yes | head -1` endet also sofort.
|
||||||
|
- Die Built-ins `pwd` und `help` lassen sich umleiten und in Pipes verwenden (`help | less`). `cd`, `set`, `export`, `exit` und `quit` verändern die Shell selbst und sind in einer Pipeline nicht erlaubt.
|
||||||
|
- Ein einzelnes `&` (Hintergrundprozesse) gibt es noch nicht, es wird als normales Zeichen übergeben.
|
||||||
|
|
||||||
### Farben
|
### Farben
|
||||||
|
|
||||||
Der Prompt wird über die Umgebungsvariable `GOSH_COLORS` eingefärbt. Sie besteht aus durch `:` getrennten Einträgen `schlüssel=code`:
|
Der Prompt wird über die Umgebungsvariable `GOSH_COLORS` eingefärbt. Sie besteht aus durch `:` getrennten Einträgen `schlüssel=code`:
|
||||||
|
|||||||
@@ -111,7 +111,16 @@ func expandTilde(path string) string {
|
|||||||
return path
|
return path
|
||||||
}
|
}
|
||||||
|
|
||||||
// splitWords splits input into words like a (very) simple POSIX shell:
|
// token is a word or an operator of a command line.
|
||||||
|
type token struct {
|
||||||
|
op string // "" for a word, otherwise |, <, >, >>, >&, <&, &> or &>>
|
||||||
|
word string // the word, if op == ""
|
||||||
|
fd int // the redirected file descriptor (for redirections)
|
||||||
|
dup int // the target file descriptor of >& and <&
|
||||||
|
}
|
||||||
|
|
||||||
|
// tokenize splits input into words and operators like a (very) simple POSIX
|
||||||
|
// shell:
|
||||||
// - words are separated by unquoted blanks, an unquoted # starts a comment
|
// - words are separated by unquoted blanks, an unquoted # starts a comment
|
||||||
// - '...' is taken literally, nothing is expanded inside
|
// - '...' is taken literally, nothing is expanded inside
|
||||||
// - "..." expands $VAR and ${VAR}; \ only escapes $ " \ and `
|
// - "..." expands $VAR and ${VAR}; \ only escapes $ " \ and `
|
||||||
@@ -120,23 +129,31 @@ func expandTilde(path string) string {
|
|||||||
// an unset variable is an error, so that e.g. "rm -rf $DIR/x" never runs
|
// an unset variable is an error, so that e.g. "rm -rf $DIR/x" never runs
|
||||||
// with an empty $DIR
|
// with an empty $DIR
|
||||||
// - ~ and ~/... are expanded at the start of an unquoted word
|
// - ~ and ~/... are expanded at the start of an unquoted word
|
||||||
|
// - unquoted |, <, >, >>, N<, N>, N>>, N>&M, N<&M, &> and &>> are operators
|
||||||
//
|
//
|
||||||
// Expanded values are never split into several words.
|
// Expanded values are never split into several words.
|
||||||
func splitWords(input string) ([]string, error) {
|
func tokenize(input string) ([]token, error) {
|
||||||
var (
|
var (
|
||||||
words []string
|
tokens []token
|
||||||
b strings.Builder
|
b strings.Builder
|
||||||
inWord bool // a word has been started (even if it is still empty)
|
inWord bool // a word has been started (even if it is still empty)
|
||||||
quote byte
|
quote byte
|
||||||
escaped bool
|
escaped bool
|
||||||
|
rawStart = -1 // start of the current word in input
|
||||||
)
|
)
|
||||||
|
|
||||||
endWord := func() {
|
endWord := func() {
|
||||||
if inWord {
|
if inWord {
|
||||||
words = append(words, b.String())
|
tokens = append(tokens, token{word: b.String()})
|
||||||
}
|
}
|
||||||
b.Reset()
|
b.Reset()
|
||||||
inWord = false
|
inWord = false
|
||||||
|
rawStart = -1
|
||||||
|
}
|
||||||
|
startRaw := func(i int) {
|
||||||
|
if rawStart < 0 {
|
||||||
|
rawStart = i
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
for i := 0; i < len(input); i++ {
|
for i := 0; i < len(input); i++ {
|
||||||
@@ -156,8 +173,10 @@ func splitWords(input string) ([]string, error) {
|
|||||||
b.WriteByte(ch)
|
b.WriteByte(ch)
|
||||||
}
|
}
|
||||||
case ch == '\\':
|
case ch == '\\':
|
||||||
|
startRaw(i)
|
||||||
escaped = true
|
escaped = true
|
||||||
case ch == '$':
|
case ch == '$':
|
||||||
|
startRaw(i)
|
||||||
value, n, err := expandVar(input[i:])
|
value, n, err := expandVar(input[i:])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -176,6 +195,7 @@ func splitWords(input string) ([]string, error) {
|
|||||||
b.WriteByte(ch)
|
b.WriteByte(ch)
|
||||||
}
|
}
|
||||||
case ch == '"' || ch == '\'':
|
case ch == '"' || ch == '\'':
|
||||||
|
startRaw(i)
|
||||||
quote = ch
|
quote = ch
|
||||||
inWord = true
|
inWord = true
|
||||||
case ch == ' ' || ch == '\t' || ch == '\n':
|
case ch == ' ' || ch == '\t' || ch == '\n':
|
||||||
@@ -183,8 +203,55 @@ func splitWords(input string) ([]string, error) {
|
|||||||
case ch == '#' && !inWord && b.Len() == 0:
|
case ch == '#' && !inWord && b.Len() == 0:
|
||||||
// Comment until the end of the line
|
// Comment until the end of the line
|
||||||
i = len(input)
|
i = len(input)
|
||||||
|
case ch == '|':
|
||||||
|
endWord()
|
||||||
|
tokens = append(tokens, token{op: "|"})
|
||||||
|
case ch == '&' && i+1 < len(input) && input[i+1] == '>':
|
||||||
|
// &> and &>> redirect stdout and stderr
|
||||||
|
endWord()
|
||||||
|
op := "&>"
|
||||||
|
if i+2 < len(input) && input[i+2] == '>' {
|
||||||
|
op = "&>>"
|
||||||
|
}
|
||||||
|
tokens = append(tokens, token{op: op, fd: 1})
|
||||||
|
i += len(op) - 1
|
||||||
|
case ch == '<' || ch == '>':
|
||||||
|
// A word consisting only of unquoted digits right before the
|
||||||
|
// operator is the file descriptor, e.g. 2>file
|
||||||
|
fd := 0
|
||||||
|
if ch == '>' {
|
||||||
|
fd = 1
|
||||||
|
}
|
||||||
|
if inWord && rawStart >= 0 && isDigits(input[rawStart:i]) {
|
||||||
|
fd, _ = strconv.Atoi(input[rawStart:i])
|
||||||
|
b.Reset()
|
||||||
|
inWord = false
|
||||||
|
rawStart = -1
|
||||||
|
} else {
|
||||||
|
endWord()
|
||||||
|
}
|
||||||
|
t := token{op: string(ch), fd: fd}
|
||||||
|
switch {
|
||||||
|
case ch == '>' && i+1 < len(input) && input[i+1] == '>':
|
||||||
|
t.op = ">>"
|
||||||
|
i++
|
||||||
|
case i+1 < len(input) && input[i+1] == '&':
|
||||||
|
// N>&M and N<&M duplicate file descriptor M
|
||||||
|
j := i + 2
|
||||||
|
for j < len(input) && input[j] >= '0' && input[j] <= '9' {
|
||||||
|
j++
|
||||||
|
}
|
||||||
|
if j == i+2 {
|
||||||
|
return nil, syntaxError(input[i : i+2])
|
||||||
|
}
|
||||||
|
t.op = string(ch) + "&"
|
||||||
|
t.dup, _ = strconv.Atoi(input[i+2 : j])
|
||||||
|
i = j - 1
|
||||||
|
}
|
||||||
|
tokens = append(tokens, t)
|
||||||
case ch == '~' && !inWord && b.Len() == 0 &&
|
case ch == '~' && !inWord && b.Len() == 0 &&
|
||||||
(i+1 == len(input) || strings.IndexByte("/ \t\n", input[i+1]) >= 0):
|
(i+1 == len(input) || strings.IndexByte("/ \t\n|<>", input[i+1]) >= 0):
|
||||||
|
startRaw(i)
|
||||||
home := getHomeDir()
|
home := getHomeDir()
|
||||||
if home == "" {
|
if home == "" {
|
||||||
home = "~"
|
home = "~"
|
||||||
@@ -192,6 +259,7 @@ func splitWords(input string) ([]string, error) {
|
|||||||
b.WriteString(home)
|
b.WriteString(home)
|
||||||
inWord = true
|
inWord = true
|
||||||
default:
|
default:
|
||||||
|
startRaw(i)
|
||||||
b.WriteByte(ch)
|
b.WriteByte(ch)
|
||||||
inWord = true
|
inWord = true
|
||||||
}
|
}
|
||||||
@@ -204,6 +272,35 @@ func splitWords(input string) ([]string, error) {
|
|||||||
return nil, fmt.Errorf("unterminated %c quote", quote)
|
return nil, fmt.Errorf("unterminated %c quote", quote)
|
||||||
}
|
}
|
||||||
endWord()
|
endWord()
|
||||||
|
return tokens, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// isDigits reports whether s is a non-empty string of ASCII digits.
|
||||||
|
func isDigits(s string) bool {
|
||||||
|
if s == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
if s[i] < '0' || s[i] > '9' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// splitWords splits input into words (see tokenize). Operators are an error.
|
||||||
|
func splitWords(input string) ([]string, error) {
|
||||||
|
tokens, err := tokenize(input)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var words []string
|
||||||
|
for _, t := range tokens {
|
||||||
|
if t.op != "" {
|
||||||
|
return nil, fmt.Errorf("unexpected operator %s", t.op)
|
||||||
|
}
|
||||||
|
words = append(words, t.word)
|
||||||
|
}
|
||||||
return words, nil
|
return words, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -469,7 +566,10 @@ func newShellCompleter() *shellCompleter {
|
|||||||
// runes of the current word that are already typed.
|
// runes of the current word that are already typed.
|
||||||
func (c *shellCompleter) Do(line []rune, pos int) ([][]rune, int) {
|
func (c *shellCompleter) Do(line []rune, pos int) ([][]rune, int) {
|
||||||
word, start, quote := currentWord(string(line[:pos]))
|
word, start, quote := currentWord(string(line[:pos]))
|
||||||
isFirstWord := strings.TrimSpace(string(line[:start])) == ""
|
// A command is expected at the start and after |, ;, && and ||
|
||||||
|
before := strings.TrimSpace(string(line[:start]))
|
||||||
|
isFirstWord := before == "" || strings.HasSuffix(before, "|") ||
|
||||||
|
strings.HasSuffix(before, ";") || strings.HasSuffix(before, "&&")
|
||||||
|
|
||||||
// Plain "~" -> "~/"
|
// Plain "~" -> "~/"
|
||||||
if word == "~" {
|
if word == "~" {
|
||||||
@@ -553,7 +653,8 @@ func currentWord(s string) (word string, start int, quote byte) {
|
|||||||
}
|
}
|
||||||
case ch == '"' || ch == '\'':
|
case ch == '"' || ch == '\'':
|
||||||
quote = ch
|
quote = ch
|
||||||
case ch == ' ' || ch == '\t':
|
case ch == ' ' || ch == '\t' || strings.IndexByte(";|&<>", ch) >= 0:
|
||||||
|
// Blanks and operators end a word
|
||||||
b.Reset()
|
b.Reset()
|
||||||
start = i + 1
|
start = i + 1
|
||||||
default:
|
default:
|
||||||
@@ -567,7 +668,7 @@ func currentWord(s string) (word string, start int, quote byte) {
|
|||||||
func escapeWord(s string) string {
|
func escapeWord(s string) string {
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
for _, r := range s {
|
for _, r := range s {
|
||||||
if strings.ContainsRune(" \t\\'\"", r) {
|
if strings.ContainsRune(" \t\\'\";|&<>", r) {
|
||||||
b.WriteByte('\\')
|
b.WriteByte('\\')
|
||||||
}
|
}
|
||||||
b.WriteRune(r)
|
b.WriteRune(r)
|
||||||
@@ -743,6 +844,14 @@ func exitStatus(err error) int {
|
|||||||
if err == nil {
|
if err == nil {
|
||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
var statusErr statusError
|
||||||
|
if errors.As(err, &statusErr) {
|
||||||
|
return int(statusErr)
|
||||||
|
}
|
||||||
|
var synErr syntaxError
|
||||||
|
if errors.As(err, &synErr) {
|
||||||
|
return 2
|
||||||
|
}
|
||||||
var exitErr *exec.ExitError
|
var exitErr *exec.ExitError
|
||||||
if errors.As(err, &exitErr) {
|
if errors.As(err, &exitErr) {
|
||||||
if ws, ok := exitErr.Sys().(syscall.WaitStatus); ok && ws.Signaled() {
|
if ws, ok := exitErr.Sys().(syscall.WaitStatus); ok && ws.Signaled() {
|
||||||
@@ -829,6 +938,21 @@ func interactive() {
|
|||||||
// lastStatus is the exit status of the last command ($?).
|
// lastStatus is the exit status of the last command ($?).
|
||||||
var lastStatus int
|
var lastStatus int
|
||||||
|
|
||||||
|
// statusError is an error that only carries an exit status; the problem has
|
||||||
|
// already been reported.
|
||||||
|
type statusError int
|
||||||
|
|
||||||
|
func (e statusError) Error() string {
|
||||||
|
return "exit status " + strconv.Itoa(int(e))
|
||||||
|
}
|
||||||
|
|
||||||
|
// syntaxError is a syntax error in the input (exit status 2, like sh).
|
||||||
|
type syntaxError string
|
||||||
|
|
||||||
|
func (e syntaxError) Error() string {
|
||||||
|
return "syntax error near unexpected token `" + string(e) + "'"
|
||||||
|
}
|
||||||
|
|
||||||
// errCommandNotFound is returned when an external command does not exist.
|
// errCommandNotFound is returned when an external command does not exist.
|
||||||
var errCommandNotFound = errors.New("command not found")
|
var errCommandNotFound = errors.New("command not found")
|
||||||
|
|
||||||
@@ -836,8 +960,11 @@ var errCommandNotFound = errors.New("command not found")
|
|||||||
// failed have already reported the problem themselves, only their exit
|
// failed have already reported the problem themselves, only their exit
|
||||||
// status is kept.
|
// status is kept.
|
||||||
func printError(err error) {
|
func printError(err error) {
|
||||||
var exitErr *exec.ExitError
|
var (
|
||||||
if err == nil || errors.As(err, &exitErr) {
|
exitErr *exec.ExitError
|
||||||
|
statusErr statusError
|
||||||
|
)
|
||||||
|
if err == nil || errors.As(err, &exitErr) || errors.As(err, &statusErr) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
fmt.Fprintln(os.Stderr, "gosh:", err)
|
fmt.Fprintln(os.Stderr, "gosh:", err)
|
||||||
@@ -872,7 +999,7 @@ func splitList(input string) ([]listItem, error) {
|
|||||||
if next == "" {
|
if next == "" {
|
||||||
next = op
|
next = op
|
||||||
}
|
}
|
||||||
return fmt.Errorf("syntax error near unexpected token `%s'", next)
|
return syntaxError(next)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -940,7 +1067,7 @@ scan:
|
|||||||
func execInput(input string) error {
|
func execInput(input string) error {
|
||||||
items, err := splitList(input)
|
items, err := splitList(input)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
lastStatus = 2
|
lastStatus = exitStatus(err)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -956,19 +1083,256 @@ func execInput(input string) error {
|
|||||||
return lastErr
|
return lastErr
|
||||||
}
|
}
|
||||||
|
|
||||||
// execCommand executes a single simple command.
|
// execCommand executes a pipeline of simple commands with redirections.
|
||||||
func execCommand(input string) error {
|
func execCommand(input string) error {
|
||||||
// Split the input into the command and its arguments, expanding
|
// Split the input into words and operators, expanding variables and ~ on
|
||||||
// variables and ~ on the way.
|
// the way.
|
||||||
args, err := splitWords(input)
|
tokens, err := tokenize(input)
|
||||||
|
var synErr syntaxError
|
||||||
|
if errors.As(err, &synErr) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("invalid input: %w", err)
|
return fmt.Errorf("invalid input: %w", err)
|
||||||
}
|
}
|
||||||
if len(args) == 0 {
|
stages, err := parsePipeline(tokens)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(stages) == 0 {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
return runPipeline(stages)
|
||||||
|
}
|
||||||
|
|
||||||
// Check for built-in commands.
|
// redirect is a redirection of a simple command, e.g. 2>>file or 2>&1.
|
||||||
|
type redirect struct {
|
||||||
|
op string // <, >, >>, &>, &>>, >& or <&
|
||||||
|
fd int // the redirected file descriptor
|
||||||
|
dup int // the target file descriptor of >& and <&
|
||||||
|
file string // the file name for all other operators
|
||||||
|
}
|
||||||
|
|
||||||
|
// stage is a simple command of a pipeline.
|
||||||
|
type stage struct {
|
||||||
|
args []string
|
||||||
|
redirects []redirect
|
||||||
|
}
|
||||||
|
|
||||||
|
// parsePipeline groups tokens into the stages of a pipeline.
|
||||||
|
func parsePipeline(tokens []token) ([]stage, error) {
|
||||||
|
var (
|
||||||
|
stages []stage
|
||||||
|
cur stage
|
||||||
|
)
|
||||||
|
isEmpty := func(s stage) bool { return len(s.args) == 0 && len(s.redirects) == 0 }
|
||||||
|
|
||||||
|
for i := 0; i < len(tokens); i++ {
|
||||||
|
t := tokens[i]
|
||||||
|
switch t.op {
|
||||||
|
case "":
|
||||||
|
cur.args = append(cur.args, t.word)
|
||||||
|
case "|":
|
||||||
|
if isEmpty(cur) {
|
||||||
|
return nil, syntaxError("|")
|
||||||
|
}
|
||||||
|
stages = append(stages, cur)
|
||||||
|
cur = stage{}
|
||||||
|
case ">&", "<&":
|
||||||
|
cur.redirects = append(cur.redirects, redirect{op: t.op, fd: t.fd, dup: t.dup})
|
||||||
|
default:
|
||||||
|
if i+1 == len(tokens) {
|
||||||
|
return nil, syntaxError("newline")
|
||||||
|
}
|
||||||
|
if next := tokens[i+1]; next.op != "" {
|
||||||
|
return nil, syntaxError(next.op)
|
||||||
|
}
|
||||||
|
i++
|
||||||
|
cur.redirects = append(cur.redirects, redirect{op: t.op, fd: t.fd, file: tokens[i].word})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if isEmpty(cur) {
|
||||||
|
if len(stages) > 0 {
|
||||||
|
return nil, syntaxError("|")
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return append(stages, cur), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// runPipeline runs the stages of a pipeline concurrently, connecting the
|
||||||
|
// stdout of each stage to the stdin of the next one. It returns the error of
|
||||||
|
// the last stage (so $? is its exit status, like in sh); errors of the other
|
||||||
|
// stages are printed.
|
||||||
|
func runPipeline(stages []stage) error {
|
||||||
|
waits := make([]func() error, len(stages))
|
||||||
|
stdin := os.Stdin
|
||||||
|
for i, st := range stages {
|
||||||
|
stdout := os.Stdout
|
||||||
|
var next *os.File
|
||||||
|
if i < len(stages)-1 {
|
||||||
|
r, w, err := os.Pipe()
|
||||||
|
if err != nil {
|
||||||
|
// Let the stages that already run finish
|
||||||
|
if stdin != os.Stdin {
|
||||||
|
stdin.Close()
|
||||||
|
}
|
||||||
|
for _, wait := range waits[:i] {
|
||||||
|
printError(wait())
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
stdout, next = w, r
|
||||||
|
}
|
||||||
|
waits[i] = startStage(st, stdin, stdout, len(stages) > 1)
|
||||||
|
stdin = next
|
||||||
|
}
|
||||||
|
|
||||||
|
var err error
|
||||||
|
for i, wait := range waits {
|
||||||
|
err = wait()
|
||||||
|
if i < len(waits)-1 {
|
||||||
|
printError(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// startStage starts a stage of a pipeline with the given stdin and stdout and
|
||||||
|
// returns a function that waits for it to finish. Pipe ends passed in are
|
||||||
|
// closed by startStage once the stage no longer needs them, so that the
|
||||||
|
// neighbors see EOF or get SIGPIPE.
|
||||||
|
func startStage(st stage, stdin, stdout *os.File, inPipeline bool) func() error {
|
||||||
|
var owned []*os.File
|
||||||
|
if stdin != os.Stdin {
|
||||||
|
owned = append(owned, stdin)
|
||||||
|
}
|
||||||
|
if stdout != os.Stdout {
|
||||||
|
owned = append(owned, stdout)
|
||||||
|
}
|
||||||
|
closeOwned := func() {
|
||||||
|
for _, f := range owned {
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result := func(err error) func() error {
|
||||||
|
return func() error { return err }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Connect the command to the terminal unless redirected. Without Stdin
|
||||||
|
// the command would read from /dev/null, so interactive programs (su,
|
||||||
|
// vim, ssh, ...) would get EOF right away. readline only reads from stdin
|
||||||
|
// while a line is being edited, so it does not steal input.
|
||||||
|
fds := [3]*os.File{stdin, stdout, os.Stderr}
|
||||||
|
opened, err := applyRedirects(&fds, st.redirects)
|
||||||
|
owned = append(owned, opened...)
|
||||||
|
if err != nil || len(st.args) == 0 {
|
||||||
|
// A command without arguments only creates or truncates the files
|
||||||
|
closeOwned()
|
||||||
|
return result(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if builtin, ok := builtins[st.args[0]]; ok {
|
||||||
|
if inPipeline && builtin.changesState {
|
||||||
|
closeOwned()
|
||||||
|
return result(fmt.Errorf("%s: cannot be used in a pipeline", st.args[0]))
|
||||||
|
}
|
||||||
|
run := func() error {
|
||||||
|
defer closeOwned()
|
||||||
|
err := runBuiltin(st.args, fds[1])
|
||||||
|
if err != nil && fds[2] != os.Stderr {
|
||||||
|
// Report the error where stderr is redirected to
|
||||||
|
fmt.Fprintln(fds[2], "gosh:", err)
|
||||||
|
return statusError(1)
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !inPipeline {
|
||||||
|
return result(run())
|
||||||
|
}
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- run() }()
|
||||||
|
return func() error { return <-done }
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd := exec.Command(st.args[0], st.args[1:]...)
|
||||||
|
cmd.Stdin, cmd.Stdout, cmd.Stderr = fds[0], fds[1], fds[2]
|
||||||
|
err = cmd.Start()
|
||||||
|
// The child has its own copies of the files now
|
||||||
|
closeOwned()
|
||||||
|
if errors.Is(err, exec.ErrNotFound) {
|
||||||
|
err = fmt.Errorf("%s: %w", sanitize(st.args[0]), errCommandNotFound)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return result(err)
|
||||||
|
}
|
||||||
|
return cmd.Wait
|
||||||
|
}
|
||||||
|
|
||||||
|
// applyRedirects applies the redirections in order to fds (stdin, stdout,
|
||||||
|
// stderr) and returns the files it opened.
|
||||||
|
func applyRedirects(fds *[3]*os.File, redirects []redirect) ([]*os.File, error) {
|
||||||
|
var opened []*os.File
|
||||||
|
fail := func(err error) ([]*os.File, error) {
|
||||||
|
for _, f := range opened {
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, r := range redirects {
|
||||||
|
if r.fd > 2 {
|
||||||
|
return fail(fmt.Errorf("%d: bad file descriptor", r.fd))
|
||||||
|
}
|
||||||
|
if r.op == ">&" || r.op == "<&" {
|
||||||
|
if r.dup > 2 {
|
||||||
|
return fail(fmt.Errorf("%d: bad file descriptor", r.dup))
|
||||||
|
}
|
||||||
|
fds[r.fd] = fds[r.dup]
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
flags := os.O_WRONLY | os.O_CREATE | os.O_TRUNC
|
||||||
|
switch r.op {
|
||||||
|
case "<":
|
||||||
|
flags = os.O_RDONLY
|
||||||
|
case ">>", "&>>":
|
||||||
|
flags = os.O_WRONLY | os.O_CREATE | os.O_APPEND
|
||||||
|
}
|
||||||
|
f, err := os.OpenFile(r.file, flags, 0666)
|
||||||
|
if err != nil {
|
||||||
|
// "file: No such file or directory" with exit status 1, like sh
|
||||||
|
var pathErr *fs.PathError
|
||||||
|
if errors.As(err, &pathErr) {
|
||||||
|
err = fmt.Errorf("%s: %v", sanitize(r.file), pathErr.Err)
|
||||||
|
}
|
||||||
|
return fail(err)
|
||||||
|
}
|
||||||
|
opened = append(opened, f)
|
||||||
|
fds[r.fd] = f
|
||||||
|
if r.op == "&>" || r.op == "&>>" {
|
||||||
|
fds[2] = f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return opened, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// builtins lists the built-in commands. Commands that change the state of
|
||||||
|
// the shell cannot be used in a pipeline, since there they would run
|
||||||
|
// concurrently (sh runs them in a subshell where they have no effect).
|
||||||
|
var builtins = map[string]struct{ changesState bool }{
|
||||||
|
"cd": {changesState: true},
|
||||||
|
"pwd": {},
|
||||||
|
"exit": {changesState: true},
|
||||||
|
"quit": {changesState: true},
|
||||||
|
"set": {changesState: true},
|
||||||
|
"export": {changesState: true},
|
||||||
|
"help": {},
|
||||||
|
}
|
||||||
|
|
||||||
|
// runBuiltin executes a built-in command, writing its output to stdout.
|
||||||
|
func runBuiltin(args []string, stdout io.Writer) error {
|
||||||
switch args[0] {
|
switch args[0] {
|
||||||
case "cd":
|
case "cd":
|
||||||
// Save current directory before changing (for 'cd -')
|
// Save current directory before changing (for 'cd -')
|
||||||
@@ -1002,7 +1366,7 @@ func execCommand(input string) error {
|
|||||||
return os.Chdir(path)
|
return os.Chdir(path)
|
||||||
case "pwd":
|
case "pwd":
|
||||||
wd, err := os.Getwd()
|
wd, err := os.Getwd()
|
||||||
fmt.Println(wd)
|
fmt.Fprintln(stdout, wd)
|
||||||
return err
|
return err
|
||||||
case "exit":
|
case "exit":
|
||||||
// exit with optional status code
|
// exit with optional status code
|
||||||
@@ -1045,7 +1409,7 @@ func execCommand(input string) error {
|
|||||||
os.Setenv(parts[0], parts[1])
|
os.Setenv(parts[0], parts[1])
|
||||||
return nil
|
return nil
|
||||||
case "help":
|
case "help":
|
||||||
fmt.Println("gosh - Go Shell\nVersion: " + versionInfo() + `
|
fmt.Fprintln(stdout, "gosh - Go Shell\nVersion: " + versionInfo() + `
|
||||||
|
|
||||||
Built-in commands:
|
Built-in commands:
|
||||||
cd [dir] - Change directory (cd with no args goes to home)
|
cd [dir] - Change directory (cd with no args goes to home)
|
||||||
@@ -1074,26 +1438,11 @@ Features:
|
|||||||
using an unset variable is an error, use ${VAR:-} for an empty default
|
using an unset variable is an error, use ${VAR:-} for an empty default
|
||||||
- Command lists: a ; b (run both), a && b (b only if a succeeded),
|
- Command lists: a ; b (run both), a && b (b only if a succeeded),
|
||||||
a || b (b only if a failed); $? is the exit status of the last command
|
a || b (b only if a failed); $? is the exit status of the last command
|
||||||
|
- Pipes (a | b) and redirections: < file, > file, >> file, 2> file,
|
||||||
|
2>> file, 2>&1, >&2, &> file (stdout and stderr), &>> file
|
||||||
- Non-interactive use: gosh -c 'command', gosh script, command | gosh;
|
- Non-interactive use: gosh -c 'command', gosh script, command | gosh;
|
||||||
the exit status is the one of the last command`)
|
the exit status is the one of the last command`)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
return fmt.Errorf("%s: not a builtin", args[0])
|
||||||
// Prepare the command to execute.
|
|
||||||
cmd := exec.Command(args[0], args[1:]...)
|
|
||||||
|
|
||||||
// Connect the command to the terminal. Without Stdin the command would
|
|
||||||
// read from /dev/null, so interactive programs (su, vim, ssh, ...) would
|
|
||||||
// get EOF right away. readline only reads from stdin while a line is
|
|
||||||
// being edited, so it does not steal input from the command.
|
|
||||||
cmd.Stdin = os.Stdin
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
|
|
||||||
// Execute the command return the error.
|
|
||||||
err = cmd.Run()
|
|
||||||
if errors.Is(err, exec.ErrNotFound) {
|
|
||||||
return fmt.Errorf("%s: %w", sanitize(args[0]), errCommandNotFound)
|
|
||||||
}
|
|
||||||
return err
|
|
||||||
}
|
}
|
||||||
+149
@@ -859,6 +859,15 @@ func TestShellCompleter_Do(t *testing.T) {
|
|||||||
{"ls " + tmpDir + "/ma", []string{"in.go"}, 2},
|
{"ls " + tmpDir + "/ma", []string{"in.go"}, 2},
|
||||||
{"ls xyz", nil, 3},
|
{"ls xyz", nil, 3},
|
||||||
{"ls ~", []string{"/"}, 1},
|
{"ls ~", []string{"/"}, 1},
|
||||||
|
// Commands are completed after |, ;, && and ||, files after redirections
|
||||||
|
{"ls | gi", []string{"tty "}, 2},
|
||||||
|
{"ls|gi", []string{"tty "}, 2},
|
||||||
|
{"true; gi", []string{"tty "}, 2},
|
||||||
|
{"true && gi", []string{"tty "}, 2},
|
||||||
|
{"false || gi", []string{"tty "}, 2},
|
||||||
|
{"ls > ma", []string{"in.go"}, 2},
|
||||||
|
{"ls 2>ma", []string{"in.go"}, 2},
|
||||||
|
{"cat <ma", []string{"in.go"}, 2},
|
||||||
}
|
}
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
got, length := completeLine(tt.line)
|
got, length := completeLine(tt.line)
|
||||||
@@ -1471,3 +1480,143 @@ func TestExecInput_CommandNotFound(t *testing.T) {
|
|||||||
t.Errorf("$? = %d, want 127", lastStatus)
|
t.Errorf("$? = %d, want 127", lastStatus)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestTokenize_Operators(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
input string
|
||||||
|
want []token
|
||||||
|
}{
|
||||||
|
{"a | b", []token{{word: "a"}, {op: "|"}, {word: "b"}}},
|
||||||
|
{"a|b", []token{{word: "a"}, {op: "|"}, {word: "b"}}},
|
||||||
|
{"a > f", []token{{word: "a"}, {op: ">", fd: 1}, {word: "f"}}},
|
||||||
|
{"a >>f", []token{{word: "a"}, {op: ">>", fd: 1}, {word: "f"}}},
|
||||||
|
{"a < f", []token{{word: "a"}, {op: "<", fd: 0}, {word: "f"}}},
|
||||||
|
{"a 2>f", []token{{word: "a"}, {op: ">", fd: 2}, {word: "f"}}},
|
||||||
|
{"a 2>>f", []token{{word: "a"}, {op: ">>", fd: 2}, {word: "f"}}},
|
||||||
|
{"a 2>&1", []token{{word: "a"}, {op: ">&", fd: 2, dup: 1}}},
|
||||||
|
{"a >&2", []token{{word: "a"}, {op: ">&", fd: 1, dup: 2}}},
|
||||||
|
{"a 0<&2", []token{{word: "a"}, {op: "<&", fd: 0, dup: 2}}},
|
||||||
|
{"a &>f", []token{{word: "a"}, {op: "&>", fd: 1}, {word: "f"}}},
|
||||||
|
{"a &>>f", []token{{word: "a"}, {op: "&>>", fd: 1}, {word: "f"}}},
|
||||||
|
// Only a word of unquoted digits is a file descriptor
|
||||||
|
{"a x2>f", []token{{word: "a"}, {word: "x2"}, {op: ">", fd: 1}, {word: "f"}}},
|
||||||
|
{`a "2">f`, []token{{word: "a"}, {word: "2"}, {op: ">", fd: 1}, {word: "f"}}},
|
||||||
|
{"a 2 >f", []token{{word: "a"}, {word: "2"}, {op: ">", fd: 1}, {word: "f"}}},
|
||||||
|
// Quoted and escaped operators are words
|
||||||
|
{`echo "a|b" 'c>d' e\<f`, []token{{word: "echo"}, {word: "a|b"}, {word: "c>d"}, {word: "e<f"}}},
|
||||||
|
// A lone & is a word
|
||||||
|
{"a & b", []token{{word: "a"}, {word: "&"}, {word: "b"}}},
|
||||||
|
{"cat ~|x", []token{{word: "cat"}, {word: getHomeDir()}, {op: "|"}, {word: "x"}}},
|
||||||
|
}
|
||||||
|
for _, tc := range tests {
|
||||||
|
got, err := tokenize(tc.input)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("tokenize(%q) returned error: %v", tc.input, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(got, tc.want) {
|
||||||
|
t.Errorf("tokenize(%q) = %+v, want %+v", tc.input, got, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecInput_SyntaxErrors(t *testing.T) {
|
||||||
|
for _, input := range []string{"echo a |", "| echo a", "echo a | | b", "echo a >", "echo a > | b", "echo a >&", "echo a 2>&x", "; a"} {
|
||||||
|
err := execInput(input)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "syntax error") {
|
||||||
|
t.Errorf("execInput(%q) = %v, want syntax error", input, err)
|
||||||
|
}
|
||||||
|
if lastStatus != 2 {
|
||||||
|
t.Errorf("execInput(%q): $? = %d, want 2", input, lastStatus)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecInput_PipesAndRedirections(t *testing.T) {
|
||||||
|
origDir, _ := os.Getwd()
|
||||||
|
defer os.Chdir(origDir)
|
||||||
|
dir := t.TempDir()
|
||||||
|
os.Chdir(dir)
|
||||||
|
|
||||||
|
read := func(name string) string {
|
||||||
|
data, err := os.ReadFile(filepath.Join(dir, name))
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("reading %s: %v", name, err)
|
||||||
|
}
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
input string
|
||||||
|
file string
|
||||||
|
want string
|
||||||
|
status int
|
||||||
|
}{
|
||||||
|
{`printf 'b\na\nc\n' | sort | head -n 2 > out`, "out", "a\nb\n", 0},
|
||||||
|
{"echo one > f1; echo two >> f1", "f1", "one\ntwo\n", 0},
|
||||||
|
{"echo one > f2; echo two > f2", "f2", "two\n", 0},
|
||||||
|
{"tr a-z A-Z < f1 > f3", "f3", "ONE\nTWO\n", 0},
|
||||||
|
{"sh -c 'echo out; echo err >&2' > f4 2>&1", "f4", "out\nerr\n", 0},
|
||||||
|
{"sh -c 'echo out; echo err >&2' &> f5", "f5", "out\nerr\n", 0},
|
||||||
|
{"sh -c 'echo err >&2' 2> f6", "f6", "err\n", 0},
|
||||||
|
// 2>&1 before >file sends stderr to the old stdout (the pipe)
|
||||||
|
{"sh -c 'echo out; echo err >&2' 2>&1 > /dev/null | cat > f7", "f7", "err\n", 0},
|
||||||
|
{"sh -c 'echo err >&2' 2>&1 | tr a-z A-Z > f8", "f8", "ERR\n", 0},
|
||||||
|
// Builtins write to redirections and pipes
|
||||||
|
{"pwd > f9", "f9", dir + "\n", 0},
|
||||||
|
{"pwd | cat > f10", "f10", dir + "\n", 0},
|
||||||
|
{"help | head -n 1 > f11", "f11", "gosh - Go Shell\n", 0},
|
||||||
|
// Errors of builtins go to the redirected stderr
|
||||||
|
{"cd /gosh/no/such/dir 2> f12", "f12", "gosh: chdir /gosh/no/such/dir: no such file or directory\n", 1},
|
||||||
|
// The status of a pipeline is the one of the last command
|
||||||
|
{"false | true > f13", "f13", "", 0},
|
||||||
|
{"true | false > f14", "f14", "", 1},
|
||||||
|
// A redirection without a command creates the file
|
||||||
|
{"> f15", "f15", "", 0},
|
||||||
|
// SIGPIPE ends the writer when the reader exits
|
||||||
|
{"yes | head -n 1 > f16", "f16", "y\n", 0},
|
||||||
|
// A command that is not found does not stop the pipeline
|
||||||
|
{"gosh-no-such-command | echo after > f17", "f17", "after\n", 0},
|
||||||
|
{"echo before | gosh-no-such-command 2> f18", "f18", "", 127},
|
||||||
|
}
|
||||||
|
for _, tc := range tests {
|
||||||
|
execInput(tc.input)
|
||||||
|
if lastStatus != tc.status {
|
||||||
|
t.Errorf("%q: $? = %d, want %d", tc.input, lastStatus, tc.status)
|
||||||
|
}
|
||||||
|
if got := read(tc.file); got != tc.want {
|
||||||
|
t.Errorf("%q: %s = %q, want %q", tc.input, tc.file, got, tc.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExecInput_RedirectionErrors(t *testing.T) {
|
||||||
|
origDir, _ := os.Getwd()
|
||||||
|
defer os.Chdir(origDir)
|
||||||
|
os.Chdir(t.TempDir())
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
input string
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{"cat < gosh-no-such-file", "gosh-no-such-file: no such file or directory"},
|
||||||
|
{"echo a > gosh-no-such-dir/f", "gosh-no-such-dir/f: no such file or directory"},
|
||||||
|
{"echo a 3> f", "3: bad file descriptor"},
|
||||||
|
{"echo a 2>&5", "5: bad file descriptor"},
|
||||||
|
{"echo a | cd /", "cd: cannot be used in a pipeline"},
|
||||||
|
{"echo a | exit 3", "exit: cannot be used in a pipeline"},
|
||||||
|
}
|
||||||
|
for _, tc := range tests {
|
||||||
|
err := execInput(tc.input)
|
||||||
|
if err == nil || err.Error() != tc.wantErr {
|
||||||
|
t.Errorf("execInput(%q) = %v, want %q", tc.input, err, tc.wantErr)
|
||||||
|
}
|
||||||
|
if lastStatus != 1 {
|
||||||
|
t.Errorf("execInput(%q): $? = %d, want 1", tc.input, lastStatus)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The command is not run if a redirection fails
|
||||||
|
if _, err := os.Stat("f"); err == nil {
|
||||||
|
t.Error("file f must not be created for a bad file descriptor")
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in new issue
Block a user