Shell Testing shell scripts

B

balanga

For many years whenever I wanted to test the reseults of a sed, find etc command I wrote a one or two line shell script and ran it to see the results, this ending up with an accumulated mess of hundreds of undocumented scripts.

After several months trying to figure out how emacs org babel tangle works I now use something like this construct:

Code: 
* display only lines without comments or having blank lines                                                                                                                                                                   

  #+begin_src sh

  sed -E '/^[[:space:]]*($|#)/d' /tmp/dnsmasq.conf > /tmp/dnsmasq.tmp                                                                                                                                                                
  #+end_src
                                                                                                                                                                                                                                                    
  #+RESULTS:                                                                                                                                                                                                                                                    

* some find commands
                                                                                                                                                                                                                                   
  #+begin_src sh
                                                                                                                                                                                                                                             
  find /tmp -type d -name 'mc*' -empty -delete
                                                                                                                                                                                               
  #+end_src
                                                                                                                                                                                                                                                    
  #+RESULTS:

Ctr-c activates the #+begin #+end block. The output is shown under #+RESULTS

this is an *.org file and you need to activete Babel languages in your emacs configuration for this to work.

It took me ages to figure this out, but I'm glad I did.
 
Hi Mate

you can improve your org file by naming the source code blocks
and naming your results blocks as well

this allows you to move the results blocks under a results header

that way you can separate the code from the results
which is much cleaner

Code: 
#+STARTUP: content
* code
** display only lines without comments or having blank lines

#+NAME: sed-comments
#+begin_src sh
sed -E '/^[[:space:]]*($|#)/d' /tmp/dnsmasq.conf > /tmp/dnsmasq.tmp
#+end_src

** some find commands

#+NAME: find-commands
#+begin_src sh
find /tmp -type d -name 'mc*' -empty -delete
#+end_src

** results
*** sed

#+RESULTS: sed-comments

*** find

#+RESULTS: find-commands

i would also reccomend looking at noweb
for literate programming

Introduction to Literate Programming

www.howardism.org www.howardism.org

Noweb Reference Syntax (The Org Manual)

Noweb Reference Syntax (The Org Manual)
orgmode.org

You can also tangle files to multiple locations as well

emacs config

github.com

freebsd-dotfiles-xps/.config/emacs at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.
github.com github.com

dotfiles

github.com

GitHub - NapoleonWils0n/freebsd-dotfiles-xps: freebsd dotfiles dell xps 15 2019

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.
github.com github.com

dotfiles org file

github.com

freebsd-dotfiles-xps/freebsd-dotfiles-xps.org at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.
github.com github.com

dotfiles org file raw


Here are my emacs notes

github.com

cerberus/emacs at master · NapoleonWils0n/cerberus

cerberus code library. Contribute to NapoleonWils0n/cerberus development by creating an account on GitHub.
github.com github.com

root dotfiles

github.com

GitHub - NapoleonWils0n/freebsd-root-xps: freebsd root dotfiles for dell xps 15 2019

freebsd root dotfiles for dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-root-xps development by creating an account on GitHub.
github.com github.com

root dotfiles org file

github.com

freebsd-root-xps/freebsd-root-xps.org at master · NapoleonWils0n/freebsd-root-xps

freebsd root dotfiles for dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-root-xps development by creating an account on GitHub.
github.com github.com

root dotfiles org file raw


changelog to keep track of git repos

github.com

changelog/README.org at master · NapoleonWils0n/changelog

changelog for git repos. Contribute to NapoleonWils0n/changelog development by creating an account on GitHub.
github.com github.com

changelog to keep track of git repos raw


Heres how i tangle my root config files


emacs playlist

www.youtube.com

emacs

emacs tips and tricks
www.youtube.com www.youtube.com
 
Many thanks for your suggestions.

I will be studying them attentively.

I'm still very much a novice with emacs. I wish I had started with it 10 years ago, and have only recently made some progress with babel which took a long time to get the hang of.
 
NapoleonWils0n said:
you can do very cool things with org babel

like having one code block call another code block
and pass a variable to a function in the second code block
Click to expand...
emacs has a very steep learning cliff. I tried giving it a try periodically over a number of years but always gave up in frustration. Eventually stumbled upon Doom which simplified things greatly although I still have a great deal to learn.

org babel seems to be hidden away and it is difficult to come across examples of it's usage if you don't know about it.

I look forward to doing some 'cool things'.

Do you have an example of passing variables between blocks?
 
NapoleonWils0n said:
Click to expand...

My head hurts when I look at that.

I'm not a black belt in babel, I'm just starting out. I'm sure a lot of brain cells were used in coming up with all that, but I'm just a learner and find it difficult to understand what is going on. In the code below, how are the two #+NAME blocks invoked?

Do you have something very basic just to get started?

ie something which creates a directory, and then something else which copies files from another directory into the one just created

Code: 
#+TITLE: Youtube Playlist Subscriptions
#+STARTUP: overview
#+STARTUP: hideblocks
* noweb code block

use yt-dlp to dump the json output of a playlist
and pipe into jq and extract the title and url and save as org link

#+NAME: noweb-block
#+begin_src sh
yt-dlp -j --flat-playlist "${url}" \
| jq -rj '"[[\(.url)][\(.title)]]\n"'
#+end_src

#+RESULTS: noweb-block

* emacs playlist
#+NAME: emacs
#+HEADER: :var url="https://www.youtube.com/playlist?list=PL7hhhG5qUoXk266pUiPhm-gqBgM5P8vBV"
#+BEGIN_SRC sh :async t :results output list :noweb yes :wrap
<<noweb-block>>
#+end_src
 
It doesnt create a directory
it outputs a youtube playlist in the org file

tangle is for writing out code to a file

This is the noweb code block

In this code block notice "${url}"
that is a variable that is passed from the second code block

Code: 
#+NAME: noweb-block
#+begin_src sh
yt-dlp -j --flat-playlist "${url}" \
| jq -rj '"[[\(.url)][\(.title)]]\n"'
#+end_src

This code block calls the noweb code block

Code: 
<<noweb-block>>

and passes the url set with the header var
to the first code block

Code: 
* emacs playlist
#+NAME: emacs
#+HEADER: :var url="https://www.youtube.com/playlist?list=PL7hhhG5qUoXk266pUiPhm-gqBgM5P8vBV"
#+BEGIN_SRC sh :async t :results output list :noweb yes :wrap
<<noweb-block>>
#+end_src

so the code that is run is

Code: 
yt-dlp -j --flat-playlist "https://www.youtube.com/playlist?list=PL7hhhG5qUoXk266pUiPhm-gqBgM5P8vBV" \
| jq -rj '"[[\(.url)][\(.title)]]\n"'

because the url variable in the noweb block is replaced with the url from the second block
 
ie something which creates a directory, and then something else which copies files from another directory into the one just created
Click to expand...

Tangling is writing code from a org source block into a file
you can create the directory structure

its not for copying files

Code: 
#+TITLE: Freebsd Dotfiles xps
#+PROPERTY: header-args :mkdirp yes :noweb yes
example tmux config
that tangles the file into 2 places

Code: 
** tmux
*** tmux config

#+NAME: tmux
#+BEGIN_SRC conf
#===============================================================================
# .tmux.conf
#===============================================================================

#===============================================================================
# vi mode
#===============================================================================

set-window-option -g mode-keys vi


#===============================================================================
# Some tweaks to the status line
#===============================================================================

set -g status-right "%H:%M"
set -g status-right-style fg=color245


#===============================================================================
# If running inside tmux ($TMUX is set), then change the status line to red
#===============================================================================

%if #{TMUX}
set -g status-bg red
%endif


#===============================================================================
# Enable RGB colour if running in xterm(1)
#===============================================================================

set-option -sa terminal-overrides ",xterm*:Tc"


#===============================================================================
# Change the default $TERM to screen
#===============================================================================

set -g default-terminal "xterm-256color"


#===============================================================================
# No bells at all
#===============================================================================

set -g bell-action none


#===============================================================================
# close panes after command has finished
#===============================================================================

set -g remain-on-exit off


#===============================================================================
# Change the prefix key to C-a
#===============================================================================

set -g prefix C-a
unbind C-b
bind C-a send-prefix


#===============================================================================
# Turn the mouse on, but without copy mode dragging
#===============================================================================

set -g mouse on


#===============================================================================
# multiple places
#===============================================================================

bind F set -w window-size


#===============================================================================
# Keys to toggle monitoring activity in a window and the synchronize-panes option
#===============================================================================

bind m set monitor-activity
bind y set synchronize-panes\; display 'synchronize-panes #{?synchronize-panes,on,off}'


#===============================================================================
# Start windows and panes at 1, not 0
#===============================================================================

set -g base-index 1
setw -g pane-base-index 1


#===============================================================================
# reload ~/.tmux.conf using PREFIX r
#===============================================================================

bind r source-file ~/.config/tmux/tmux.conf \; display "Reloaded!"


#===============================================================================
# default statusbar colors
#===============================================================================

set -g status-style bg=default,fg=yellow #yellow


#===============================================================================
# default window title colors
#===============================================================================

set -g window-status-style fg=brightblue,bg=default


#===============================================================================
# active window title colors
#===============================================================================

set -g window-status-current-style fg=black,bg=blue


#===============================================================================
# pane border
#===============================================================================

set -g pane-border-style fg=black #base02
set -g pane-active-border-style fg=black #base01


#===============================================================================
# message text
#===============================================================================

set -g message-style bg=black,fg=brightred #orange


#===============================================================================
# pane number display
#===============================================================================

set-option -g display-panes-active-colour blue #blue
set-option -g display-panes-colour brightred #orange


#===============================================================================
# clock
#===============================================================================

set-window-option -g clock-mode-colour green #green


#===============================================================================
# vim key bindings
#===============================================================================

setw -g mode-keys vi
bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R
bind-key -r C-h select-window -t :-
bind-key -r C-l select-window -t :+


#===============================================================================
# resize panes using PREFIX H, J, K, L
#===============================================================================

bind H resize-pane -L 5
bind J resize-pane -D 5
bind K resize-pane -U 5
bind L resize-pane -R 5


#===============================================================================
# copy and paste
#===============================================================================

set-window-option -g automatic-rename on


#===============================================================================
# toggle statusbar
#===============================================================================

bind-key s set -g status


#===============================================================================
# copying selection vim style
#===============================================================================

bind-key Escape copy-mode            # enter copy mode; default [
bind-key p paste-buffer                # paste; (default hotkey: ] )
bind-key P choose-buffer             # tmux clipboard history
bind-key + delete-buffer \; display-message "Deleted current Tmux Clipboard History"


#===============================================================================
# Note: rectangle-toggle (aka Visual Block Mode) > hit v then C-v to trigger it
#===============================================================================

bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -T copy-mode-vi V send-keys -X select-line
bind-key -T copy-mode-vi C-v send-keys -X rectangle-toggle
bind-key -T choice-mode-vi h send-keys -X tree-collapse
bind-key -T choice-mode-vi l send-keys -X tree-expand
bind-key -T choice-mode-vi H send-keys -X tree-collapse-all
bind-key -T choice-mode-vi L send-keys -X tree-expand-all
bind-key -T copy-mode-vi y send-keys -X copy-pipe-and-cancel "wl-copy && wl-paste -n | wl-copy -p"
bind-key p run "wl-paste -n | tmux load-buffer - ; tmux paste-buffer"


#===============================================================================
# tmux auto rename pane
#===============================================================================

set-option -g status-interval 1
set-option -g allow-rename on
set-option -g automatic-rename on
set-option -g automatic-rename-format "#{?#{==:#{pane_current_command},zsh},#{b:pane_title},#{pane_current_command}}"


#===============================================================================
# tmux title program name
#===============================================================================

set-option -g set-titles on
set-option -g set-titles-string "#W"


#===============================================================================
# resize
#===============================================================================

setw -g aggressive-resize on
set -g focus-events on


#===============================================================================
# ssh-agent
#===============================================================================

set-environment -g SSH_AUTH_SOCK "$XDG_RUNTIME_DIR/ssh-agent.socket"
#+END_SRC

*** tmux tangle
**** home dir

#+NAME: tmux-home-dir
#+BEGIN_SRC conf :tangle "~/.config/tmux/tmux.conf"
<<tmux>>
#+END_SRC

**** current dir

#+NAME: tmux-current-dir
#+BEGIN_SRC conf :tangle ".config/tmux/tmux.conf"
<<tmux>>
#+END_SRC

github.com

freebsd-dotfiles-xps/freebsd-dotfiles-xps.org at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.
github.com github.com

 
I thought that the best way to experiment with babeling was to create a dedicated jail which could easily be reproduced at any time.

My clumsy attempt at doing something like this looks like:-

Code: 
#install TESTBED for babeling

#+begin_src sh /tmp/TESTBED-install.sh

JAIL='TESTBED'

cat <<EOF > /tmp/$JAIL.json
{
    "pkgs": [
    "emacs",
    "zsh",
    "tmux"
    ]
}
EOF

iocage create -r latest -p /tmp/$JAIL.json -n $JAIL vnet=on dhcp=on
#+end_src

This actually creates my TESTBED jail but does not install the pkgs.

There's plenty to improve but that is the basic idea.
 
Thats completely wrong

you arent tangling the file
pointless use of cat

and you wouldnt run a command like iocage
in a source block because that will lock up emacs until it completes

Code: 
#+TITLE: Jail
#+PROPERTY: header-args :mkdirp yes :noweb yes

#+NAME: jail
#+begin_src json :tangle "/tmp/TESTBED.json"
{
    "pkgs": [
    "emacs",
    "zsh",
    "tmux"
    ]
}
#+end_src

tangle the code block above
by putting your cursor on #+NAME: jail and running the the following key strokes

#+begin_example
C-u C-c C-v t
#+end_example

that will create the following file

#+begin_example
/tmp/TESTBED.json
#+end_example
 
T-Daemon said:
Try following package name format:
Code: 
cat <<EOF > /tmp/$JAIL.json
{
    "pkgs": [
        "editors/emacs",
        "shells/zsh",
        "sysutils/tmux"
    ]
}
EOF
Click to expand...
That has no bearing on things.

This works normally.

Code: 
JAIL='TESTBED'
cat <<EOF > /tmp/$JAIL.json{                                                                                                                                                                                                         
    "pkgs": [
    "emacs",
    "zsh",     
    "tmux"      
    ]         
}
EOF       
iocage create -r latest -p /tmp/$JAIL.json -n $JAIL vnet=on dhcp=on

The problem is with the babeling in the org sections.
 
I tried religiously following the instructions where you showed how to write demo script (around the 14th minute mark)

Code: 
#+TITLE: demo script                                                                                                                                                                                                                                                     
#+STARTUP: content                                                                                                                                                                                                                                                       
#+STARTUP: overview hideblocks                                                                                                                                                                                                                                           
#+OPTIONS: num:nil author:nil                                                                                                                                                                                                                                           
#+PROPERTY: headerargs :makedirp yes                                                                                                                                                                                                                                     
* tangle dotfiles                                                                                                                                                                                                                                                       
*** tangle document                                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                        
    C-u C-c C-v t                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                        
*** tangle only one code block                                                                                                                                                                                                                                           
*** tangle from command line                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                        
    #+NAME: demo-script                                                                                                                                                                                                                                                 
    #+begin_src sh                                                                                                                                                                                                                                                       
    #!/bin/sh                                                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                        
    echo 'emacs kicks vim arse up and down the street and back again'                                                                                                                                                                                                   
    #+end_src

I had no idea how to approach entries ubuntu root dotfiles etc, so was not successfull in running this script.

There is just so much to learn with all this stuff.
It seems far too complicated for me. I'm just a beginner and need something very straight forward to build on.
 
To tangle the code you just need to put your cursor on the #+NAME block
and then run C-u C-c C-v t

that mean hold control
then press u c v

then press t without control

everyone was a beginner once
emacs isnt easy, the only way is reading all the documentation

you could look at using Makefile instead

 
I think I have made some progress with my file and have managed to build my TESTBED jail somehow, but would incorporate some way of using my zshrc and alias as well as doom emacs to create an environment I am comfortable with.

I can see a long journey ahead of me. At least I've made the first step.
 
You must log in or register to reply here.
Back
Top