Why crucible?

So the idea was to first name it anvil or ansible but those two names already exist in the DevOps world as other tools. So then I came up with the idea of a framework that can mould things and then you pour it like from a crucible into whatever shape you chose to make and replicate the scripts completely. In essence I wanted to make a virtualenv for shell scripts.

The obvious question is why do I wanted to make this. So the reason was mainly to have a nice framework for repeatable and quickly made shell scripts. All those manipulations that you might forget over time and only occasionally need. Like arrays in Bash, or date times, or navigation. There exist a collection of Bash functions but it was not nearly as nicely structured and you had to manually source them.

In addition to this all, I just wanted to see how far I can take one language that is designed to do a very limited set of things. See how far I can take this limited set and make it do things that are normal in other programming languages.

Structure

The structure of the side project is that for each of the shells I want to target:

• Bash
• Fish
• ksh
• csh
• zsh

I will have written functions that you can require. The entrypoint of the main shell script is a file called .mould . It contains the require function logic. Basically it will download the requirements and source them into your current script.

Then when you are done you call the pour command and it will produce a tarball that will hold the full script with dependencies so you can run it anywhere and everywhere.

Example functions

For example I made some functions pertaining to dealing with arrays in the shell. There even exists a map functionality that applies a function to every element of the array and thereby transforming the array in place. There exists some functions to deal with dates and logging.

Test harness

There is a test suite in place that just unit tests the functions to see if the input / output is what is expected. This was quite the thing to setup and maintain actually. Every script is checked with Shellcheck to see if it passes static code analysis.

Open Source

It is open source and the repository is located over here. There are some releases now there but they are haphazard at best.

Future

The future will contain that there will be modules for text, binary, files, network and processes and some more shells. Things like file navigation, file creation for example will all be there.

#code #bash

Getting started

To get started and use it go download the binary and save it somewhere in your PATH in your favourite shell. Then start up your terminal with the shell in question. Navigate to a new path, in our case ~/projects/crucible-example. Within that directory run crucible init. The script will ask two questions. What will be the name of the project and what will the name of the main file be. Project will be Tutorial and the main file will be run.sh. Then your directory will look a bit like this:

~/projects/crucible-example
 |
  -- .crucible
  -- .mould
  -- run.sh

The contents of .crucible is :

working_dir=~/projects/crucible-example
shared_dir=~/.local/share/crucible/4e800f0bd0ef69a050bbcae2e0f10c58d058920ce6a686441cfc07011ad7dd16
version=0.2.6
location=v0.2.6
project_name=Tutorial
main_file=run.sh
author=StealthyCoder
created=1595939715

The shared_dir is a place that will follow the XDG conventions if they are set in place and it will house the modules that will be downloaded (required) later on. This will be sectioned off and versioned so you can work on multiple projects without worrying about clashes in versions.

Then .mould will contain the logic of the require function.

Simple example

In our case let us add some logic to the run.sh file. If you open it up you see the following:

#!/usr/bin/env bash

source .mould

This is the start. We shall add the following code to it making it look as follows:

#!/usr/bin/env bash

source .mould
require arrays/arrays
require logging/logging

arrays.transform_into_array "a"
arrays.add_all a 1 2 3 4

function multiply_by_2 {
    local result
    result="$(( $1 * 2))"
    echo "$result"
}

arrays.map a multiply_by_2

function simple_log {
    logging.info "The value is: $1"
}

arrays.foreach a simple_log

Explanation of the code

require

The two require lines get the modules downloaded to where your shared location is in the .crucible file.

arrays.transform_into_array

This line will create a global variable with the name you give it and make it be an array.

arrays.add_all

This line will add to the newly created array all the values after the array name. The first position is the name of the array and there are checks and validators that do in fact check if the variable is an array or not.

arrays.map

The function declaration above it is just there to have a function that will get an input and return that multiplied by 2. The arrays.map takes an array and then the name of a function to be applied to each element. It will check if the arguments are indeed an array and a function. The result of the function will be stored at that position in the array. In our case all numbers will be doubled.

arrays.foreach

The function declaration above it is just there to have a simple logging function that states "The value is: " with the value given to it. This does not change the array in place however.

Getting a tarball

So this script is nice but it is not distributional, yet. Running crucible pour will get us a tarball with everything correctly in it. The name of the tarball is the name of the project. In this case Tutorial.tar.gz . Within are two files: .lib and run.sh. The .lib file contains all the functions needed and the run.sh just has the require lines removed and the source .mould turned into source .lib.

Improvements

The improvements I want to make on the pour command is that it applies so called tree shaking. Meaning it will only contain what is actually called in from within the script for example, but also make it so it does not have the functions double declared as is the case now.

#code #bash

Cache

I want to implement a caching mechanism for both in-memory and file based cache. The in-memory will just be an associative array with some functionalities in there. The file based one will have the associative array be declared and written to a temp file. It will feature timer based expiry component that checks if it should be evicted. Auto fetching a cache maybe, if it is used when consuming an API.

This all can be built on top of the just completed maps module.

Classes

I want to make a module for classes. They will not really be classes but sort of declared variables and a list of class definitions and class instances. However I think it will be quite fun and funny to see this in action. It will also contain a method called ctor which will instantiate things, then you can always just put down:

class_instance.a=1

Like in Python and now it will have that property.

HTTP framework

It would be fun to make a http module that will be able to make requests. It would make get requests with wget maybe but actually send POST requests with the use of nc.

Web framework

To take it a little further what I want to try and make is a Unix domain socket and have it hook into a live running Bash process that will read it and execute controllers based on the routes. It should make use of the processes, io , http and class modules.

#code #bash