Generated 2022-04-26 04:10:10 PM
This is documentation on the scripts and related information in the _ebDev repository. This document is made on a hobby basis and may have errors, but I try to make it stylistically consistent, complete, and accurate.
The major sections of this document are:
I use the term "scripts" more broadly than its actual meaning. While the great majority of files in this repository are computer scripts to be run via language interpreters such as Processing, Python, bash, not all of them are. Some are technical or configuration resources, notes etc., and documentation comments may be collated from those as well.
This document is available in these formats: Markdown - HTML - Open Document Text - PDF - Word
See the USAGE comments in makeDocumentation.sh for details, but in summary, it collates and parses documentation from comments in scripts, and combines that with a header and footer document into this larger document.
This is for convenience, but if you find a discrepancy between described behavior from anything collated here and actual script behavior, consult the documentation comments in a script itself. It may be that this document hasn't been updated after a script was updated, as the repository can be in high flux. To quickly locate and open a script, I usually use the Everything Search Engine on Windows (with a hotkey set to open the search window from anywhere) or CMD+P (file finder) in the Atom text editor on Mac, then open it for editing. (For Windows I assocaite script file extensions with a text editor so that double clicking it opens it to edit.) Also, if you don't find documentation for a script here, the script may be newer than the last time this document was regenerated.
Documentation for any script is presented after a header which gives the path to the script relative to the /scripts subdirectory.
Also, scripts may be listed in this ordering or rank:
.sh for bash scripts or .py for Python scripts, by order of what I like best for creative coding purposes first.setFileTimestampsToEarliestGitCommit.sh.The result may be a hybrid rank of what type of script I find most interesting and/or most recently modified.
Also, collated documentation comments may refer you to functional code below the comments, but that code is not copied into this document (only documentation comments are). You must find and open the source code files themselves to look at the functional source code.
Interactive demonstration of detection of whether lines intersect. Tweaked from:
http://www.jeffreythompson.org/collision-detection/line-line.php
Run the sketch. Move the mouse to move the end of the 2nd line so that it crosses or does not cross the first line, and see what happens.
Function that creates an ArrayList of ArrayLists of all possible combinations of something (this example uses ints, but you can use anything that a one-dimensional ArrayList can contain).
Copy and paste and adapt the code for your purposes. See more detailed documentation after the CODE comment.
Generates points of a Vogel spiral (Fibonacci spiral variant), and animates attributes of the dot (what is animated may vary with different versions of this script that I might make, and different parameters I may give it).
dawesome toolkit (Processing library)
Install Processing, open this file in the Processing editor or a Processing script runner, and run the script.
Prints interpolation values, and displays color swatches for, colors interpolated at start color 'from' to end color 'to', in N steps, using Processing's built-in lerpColor function. Hack the GLOBAL VARIABLES (see that comment) to change what is interpolated. Image size depends on how many colors you hard-code. For hacking/finding color gradients.
Open this file in the Processing IDE (or anything else that can run a Processing file), edit the assignments to the "from" and "to" variables to colors of your choosing, and run it with the triangle "Play" button.
NOTES:
Prints variants of constructed random character sets (hard-coded but hackable: block characters), one complete constructed screen of them after another, with character color morph (randomization).
If you hack this to display thousands of frames per variation, it slows down.
Babel Stone True Type Font from: https://www.babelstone.co.uk/Fonts/Shapes.html
or my copy of it from:
http://earthbound.io/data/dist/BabelStoneShapes.ttf
also to toy with:
http://earthbound.io/data/dist/bitBlocks.ttf
This is in development and not necessarily to my liking yet. The functionality may vary from this description in development or even finally. Displays 4 glyphs from a set of 16, in a 4-tile, in a sequence all possible combinations of the tiles, repetition allowed, each tile rotating in 90 degree increments. 4 tiles * 4 rotations each = 16 possible tile views, 16 views pick 4 all possible combos allow repeat = 65535 4-tile views.
Open this file in the Processing IDE (or anything else that can run a Processing file), and run it with the triangle "Play" button.
Rapidly generates ideas for random squiggle lines, which are random coordinates connected by curves. May be useful as an art idea brainstorming tool when I add image/svg save functionality, which I plan to do. Interesting just to watch though.
Open this file in the Processing IDE (or anything else that can run a Processing file), and run it with the triangle "Play" button.
This is in development, and not necessarily to my liking yet. Draws random straight lines connecting clusters of vertices in randomly arranged cells.
Open this file in the Processing IDE (or anything else that can run a Processing file), and run it with the triangle "Play" button.
This is in development and not useful yet.
Creates color observation fields of all possible permutations of a list of colors (choosing 3 colors from the list), after a field model diagram I found for the CIECAM02 color observation model. At this writing, it renders all of them very rapidly and quits; it will be useful when it saves svgs and/or pngs of them.
Renders a random rugged mountain line of varying slopes.
Renders a random jagged mountain line of 45 or 135 deg. slopes only.
"By Small and Simple Things"
Generates an animated grid of playfully colored concentric nested circles and/or n-gons which change on user click/drag. See USAGE for details and functionality.
Pretty closely imitates art by Daniel Bartholomew, but with random generative constraints: https://www.abstractoons.com/2017/05/03/by-small-and-simple-things/
ON INSTALLATION, if you intend to tweet, you must copy twitterAPIauth.txt (from one folder up) into the /data subfolder, and decrypt it so it actually gives the API auth, uh, thingies.
Random walk with drawn primary shapes (with outlines) of colors randomly chosen from palette. Evolved from rnd_walk_draw_square.pde
Open this file in the Processing IDE (or anything else that can run a Processing file), and run it with the triangle "Play" button.
Random walk of drawn squares with heavy outline and fills of random colors. Heavily modified from Random Walker example, "The Nature of Code," by Daniel Shiffman, http://natureofcode.com
Open this file in the Processing IDE (or anything else that can run a Processing file), and run it with the triangle "Play" button.
Rapidly generates and saves random irregular geometry ideas.
Gets -n chromatic intensities (different states of saturation) for any -c color via the CIECAM02 color space. Writes results to a new .hexplt file named after the color.
Python and the various import libraries declared at the start of CODE.
Run this script through a Python interpeter with the --help parameter for instructions, or read the description sections in the argsparse setup below.
EXAMPLE RUN that produces 16 chromacities of medium blue, which the script will write to the file named 0041ff_6_chromas.hexplt:
python /path/to_this_script/getNchromasOfColorCIECAM02.py -n 7 --COLOR 0041ff fff585_15shades.hexplt.Prints N shades of gray (parameter 1) as calculated in the RGB color space, only for comparison to any grayscale palette generated by getNshadesOfColorCIECAM02.py. You don't want to use the output of this script, because RGB math doesn't align well with human perception of light/dark/nearness of colors. You want to use getNshadesOfColorCIECAM02.py, because the color space it uses is superior for perceptual grayscale and color shade palette generation.
Run this script through a Python interpeter with one parameter, which is the number of shades of gray you want to print in hex-encoded RGB values. For example, this will print 16 shades of gray:
python /path/to_this_script/getNshadesOfGrayRGB.py 16 To write the printout to a file, use the > operator, like this:
python /path/to_this_script/getNshadesOfGrayRGB.py 16 > 18shadesOfGrayRGB.hexplt Creates all possible combinatations of colors of subset size N from an RGB hex palette file (.hexplt format), and writes them to a subfolder and files named after: the palette, the size of the subset (N), and the number for the combination in the sequence of all possible combinations.
Python 3 with itertools module installed.
This script overwrites any target files that already exist without warning.
Run this script through a python interpreter, with these parameters:
For example, to calculate and write all possible 3-combinations from the file RAHfavoriteColorsHex.hexplt, run:
python path/to_this/palette_combinations.py RAHfavoriteColorsHex.hexplt 3 Compares two palettes (.hexplt format, from parameter 1 and 2) and prints a dissimilarity ranking float between 0 and 1 (zero and one hundred percent). See USAGE for how it works.
Run this script through a Python interpreter, with these parameters:
Example that will result in a print of a similarity ranking percent (a float between 0 and 1) for two palettes:
python /path/to_this_script/paletteCompareCIECAM02.py FD613A_F9C956_BAB25B_3D9676_1F463E.hexplt FE6737_F8D845_9DC17E_489B73_375557.hexplt Tests all possible --GROWTH_CLIP rates of color_growth.py, by running it repeatedly, saving .cgp presets for each of the same.
Run this script through a Python interpreter, without any parameters:
python /path/to/this/script/color_growth_clip_tests.py This assumes that color_growth.py is in your PATH, and searches for it (which can cause a delay at the start). If you're in a hurry, hard-code the pathToColorGrowthPy variable to the correct path, and comment out the path search code.
Loads an arbitrary image file (sys.argv[1]) and gets random pixel coordinates with pixel colors, and constructs a parameter switch name and string of them (and other needed switches and their values) to pass to color_growth.py. (using --CUSTOM_COORDS_AND_COLORS) Prints this to stdout. May be captured by other scripts to be further used (e.g. to actually pass to color_growth.py).
I recommend that you call this from the script call_get_rnd_CCC_for_color_growth-py.sh (SEE). Run this script through a Python interpreter, with these positional parameters:
Example run:
python /path/to_this_script/get_rnd_CCC_for_color_growth.py inputImageFileName.png Results are printed to stdout, and may be captured e.g. by bash and passed to color_growth.py this way; the following assumes that this script and getFullPathToFile.sh are both in your PATH:
pathToScript=$(getFullPathToFile.sh get_img_RND_CCC_for_color_growth.py | sed 's/get_img_RND_CCC_for_color_growth: \(.*\)/\1/g')
var_CUSTOM_COORDS_AND_COLORS=$(python $pathToScript inputImageFileName.png) Prints detailed python platform/version information.
Run from a Python interpeter without any parameters to the script:
python /path/to/this/script/printPythonVersionDetails.py Prints information about a Python module, such as functions. re: https://stackoverflow.com/a/31005891
Run through the Python interpreter, with this script as the first parameter (to python) and the script to analyze as the second parameter:
python /path_to/this_script/analyse_module.py module_script.py Prints N variants of constructed random character sets (hard-coded but hackable: block
characters), at X characters across and Y lines down each. Prints to either terminal or
files; hack the global variable SAVE_TO_RND_FILENAMES to alter that; hack the other globals
also for whatever other purposes you might have.
SEE ALSO randomNSetChars.pde.
Python 3.8 (or maybe any 3.x version) with random and time modules installed. Moreover, python may need to be compiled with UCS2 or UCS4 support (larger text code pages support).
Run from a python interpreter:
python /path/to_this_script/randomNsetChars.py To pipe what is printed to a file, run for example:
python /path/to_this_script/randomNsetChars.py > 1000000_randomNsetCharsVariants.txt writes all possible combinations of four base-16 digits (allowing repetition of digits) to all16products.txt.
Run this script through a Python interpreter withuot any parameters:
python /path/to_this_script/helper_all_16_permutations.py Extract N (parameter 2) colors from an image (parameter 1), via Python colorgram library, and print them as RGB values.
Run this script through a python interpreter with these parameters:
For example, if the source image is named source.png, and you want to extract 23 colors, run:
python full/path_to_this_script/get_colors_colorgram.py darks-v2.png 23 To pipe the results to a text file, add a > redirect operator and text file to the end of that command, like this:
python full/path_to_this_script/get_colors_colorgram.py darks-v2.png 23 > darks-v2.rgbplt Extracts color samples from an image (parameter 1), divided into a grid of columns and rows, with each color sample taken from the color of each cell in the grid. Prints the samples as RGB values in hexadecimal format.
python PIL and numpy libraries
Run this script through a Python interpreter, with these parameters:
For example, if the source image is named darks-v2.png, and you want to sample from the center of each cell in a grid 16 across (16 columns) and 12 down (12 rows), run:
python full/path_to_this_script/get_color_sample_grid_sRGB.py darks-v2.png 16 12 To pipe the results to a text file, add a > redirect operator and text file to the end of that command, like this:
python full/path_to_this_script/get_color_sample_grid_sRGB.py darks-v2.png 16 12 > darks-v2.hexplt Creates a list of colors expressed as RGB hexadecimal values, from a simplified CIECAM02 color space. Names the output file partly after the number of elements in the list. Variant of script that uses one ciecam02 python library; this uses colorspacious, which at this writing I seem not to understand (or maybe it had errors) and it sometimes produces very out of place colors.
python, with these packages: numpy, colorspacious, colormap, more_itertools.
Run this script through a Python interpreter, without any parameters:
python /path/to_this_script/get_CIECAM02_simplified_gamut.py Produces a file named
To change the number of output colors, hack the scripts' variable assignments to J_step, C_step and h_step.
Gets -n shades of any -c color (default white if not passed) via the CIECAM02 color space. Writes results to a new .hexplt file named after color. SEE ALSO get_color_gradient_culori.js. That and this might both do hue shifting (you may not get perceptual hue constancy) in shades (~culori possibly less). For example you might say that shades of yellow look orange. But with shades of gray, this produces results that seem to align more with my idea of neutral gray (between black and white, as seen from optical blending experiments).
Python and the various import libraries declared at the start of CODE.
Run this script through a Python interpeter with the --help parameter for instructions, or read the description sections in the argsparse setup below.
EXAMPLE RUN that produces 16 shades of gray (as it defaults to shades of white if no --COLOR is specified), which the script will write to the file named FFFFFF_12shades.hexplt:
python /path/to_this_script/getNshadesOfColorCIECAM02.py -n 12 EXAMPLE RUN that produces sixteen shades of magenta, which the script will write to a file named FF00FF_18shades.hexplt:
python /path/to_this_script/getNshadesOfColorCIECAM02.py -n 16 --COLOR FF00FF get_color_gradient_culori.js over this.fff585_15shades.hexplt.Prints high brightness hues from the CIECAM02 color model in N (parameter -n) steps from 0 to 360 degrees of hue. Colors are printed as RGB colors in hex format.
Run this script through a Python interpreter, and pass the script a numeric value for the switch '-n', which is the number of full chroma hues you wish to print. For example, to print 18 hues, run:
python /path/to/this/script/printNhuesCIECAM02.py -n 18 To write the colors to a .hexplt file for permanent palette storage (recommended), use the > operator:
python /path/to/this/script/printNhuesCIECAM02.py -n 18 > 18_max_chroma_hues_from_CIECAM02.hexplt Takes a list of RGB colors expressed in hex (one per line) and sorts them using an advanced color appearance model: CIECAM02, sorting into lists of darks and lights (nearest to black and nearest to white), via the colorspacious library. Adapted from RGBhexColorSortInCIECAM02.py.
From a directory with a .hexplt file, run this script through a Python interpreter, with a .hexplt list as the first (and only) parameter to the script; for example:
python /path/to_this_script/RGBhexColorSortToDarksAndLightsIn2CIECAM02.py inputColors.hexplt It will print the results to new files named darks.hexplt and lights.hexplt. You may sort those results by next nearest color via e.g. RGBhexColorSortInCIECAM02.py.
Variant of RGBhexColorSortInCIECAM02.py which uses the color-science library and CAM16 color model for color sorting. In my tests, RGBhexColorSortInCIECAM02.py (which uses the colorspacious library) does better for hue matching independent of brightness; this (which uses colour-science) does better for matching first brightness/saturation, then hue. Both give good results. To my eye colorspacious is better overall (maybe I reckon hue before brightness/saturation). HOWEVER, rgbHexColorSortInOkLab.js is blazing fast compared to both of them, and may produce better results than both. It's a subjective call.
Python 3, various python packages (see import list at start of code) which you may install with easy_install or pip; NOTE that the colour import is actually from the colour-science package; you must install colour-science.
Run this script through a Python interpreter, with these positional parameters:
Example run with only a hexplt file given:
python /path/to_this_script/RGBhexColorSortInCAM16-UCS.py PrismacolorMarkers.hexplt The result will be printed to stdout, so that you may for example redirect the output to a new .hexplt file, e.g.:
python /path/to_this_script/RGBhexColorSortInCAM16-UCS.py inputColors.hexplt > result.hexplt Example run with the optional second parameter, which will cause the script to overwrite the original .hexplt file with the sorted version of it:
python /path/to_this_script/RGBhexColorSortInCAM16-UCS.py inputColors.hexplt foo Example run with the optional third parameter, an arbitrary color to start comparisons with (here, magenta):
python /path/to_this_script/RGBhexColorSortInCAM16-UCS.py inputColors.hexplt foo #ff00ff input file, or any other unexpected data, it may stop with an unhelpful error.
https://colour.readthedocs.io/en/develop/generated/colour.RGB_to_XYZ.html
Samples every pixel in an image and dumps to a list of hex color codes contained in the image.
Run this script through a Python interpreter, with the image file name as the first parameter to this script. Example:
python path/to_this/img2RGBhexList.py input_image.png To pipe the output to a file:
python path/to_this/img2RGBhexList.py input_image.png > input_image_hex_colors_list.hexplt To reduce that to unique colors:
sort tmp.hexplt > tmp_garblethax.txt && uniq tmp_garblethax.txt > tmp.hexplt && rm ./tmp_garblethax.txt Reworks an ArtRage script file to export a new image after every "brush stroke." These may be strung together e.g. with ffmpegAnim.sh to create a video (animation) of the progress of a created painting from start to finish.
Run via python and pass one parameter, which is an ArtRage script file, e.g.:
python /path/to_this_script/artRage2imgs.py inputArtRageScriptFile.arscript Takes a list of RGB colors expressed in hex (one per line) and sorts them
using an advanced color appearance model: CIECAM02.
If you're sorting very different colors, YOU MAY WISH to use
RGBhexColorSortinCAM16-UCS.py instead (see). This uses the colorspacious library; that uses colour-science. HOWEVER, rgbHexColorSortInOkLab.js is blazing fast compared to both of them, and may produce better results than both. It's a subjective call.
Python (probably Python 3.5), various python packages (see import list at start of code) which you may install with easy_install or pip.
Run this script through a Python interpreter, with these positional parameters:
Example run with only a hexplt file given:
python /path/to_this_script/RGBhexColorSortInCIECAM02.py PrismacolorMarkers.hexplt The result will be printed to stdout, so that you may for example redirect the output to a new .hexplt file, e.g.:
python /path/to_this_script/RGBhexColorSortInCIECAM02.py inputColors.hexplt > result.hexplt Example run with the optional second parameter, which will cause the script to overwrite the original .hexplt file with the sorted version of it:
python /path/to_this_script/RGBhexColorSortInCIECAM02.py inputColors.hexplt foo Example run with the optional third parameter, an arbitrary color to start comparisons with (here, magenta):
python /path/to_this_script/RGBhexColorSortInCIECAM02.py inputColors.hexplt foo #ff00ff input file, or any other unexpected data, it may stop with an unhelpful error.
See comments of get_CIECAM02_simplified_gamut.py. This is that but without modifying j and c; this has full brightness and chroma hues.
Run this script through a Python interpreter, without any parameter:
python /path/to/this/script/get_full_c_bright_h_jch_simplified_palette.py It creates output files named:
<h_step>_full_c_bright_h_jch_simplified.gamut
<h_step>_full_c_bright_h_jch_simplified_to_RGB.hexplt Where h_step is an integer determined by the value of the h_step variable.
Creates a list of colors expressed as RGB hexadecimal values, from a simplified CIECAM02 color space. Optionally (with a tweak described under USAGE) prints CIECAM02 JCH components instead.
python, with these packages: numpy, ciecam02, colormap, more_itertools
Run through a Python interpreter, without any parameters:
python /path/to/this_script/get_CIECAM02_simplified_gamut.py To pipe the results to a new file, do this:
python /path/to/this_script/get_CIECAM02_simplified_gamut.py > CIECAM02_simplified_gamut.gamut But you will probably get out-of-gamut warnings (out of range) if you convert to RGB with this--you may ignore them in terms of the script running (you will still get valid output). To suppress error prints by piping them to /dev/null in a Unix environment, but still get the valid hex color code printout, run it this way:
python /path/to/this_script/get_CIECAM02_simplified_gamut.py > oot.txt 2>/dev/null You may alternate comments after the OUTPUT OPTIONS comment for either JCH (simplified CIECAM02 gamut) or RGB HEX output.
See scripts with 'get', 'gamut', and '.py' in their file names. This is adapted for RGB, and the output is horrible and garish, as is RGB math.
Run through a Python interpreter without any parameters:
python path/to/this_script/get_simple_RGB_gamut.py Prints RGB hex code values (in format #dddddd) for colors in hue (LCH) steps of N, from 0 to 360 (N is hard-coded to 7; hack the script to change that), via spectra library. Luminescense and Chroma are hard-coded to 70 (of 100) and 50 (of 100); also hack the script to alter those.
Run through a python interpreter without any parameters to this script:
python /path/to_this_script/print_hcl_saturated_color_wheel_steps.py Creates a list of colors expressed as RGB hexadecimal values, from a simplified LCH (a.k.a. HCL) gamut. Capture the list with the > operator via terminal (see USAGE). Can be hacked to do this with several gamuts which the supporting "spectra" python library can use. To change the step over various components in the gamut, hack the A_domain, B_domain and C_domain variables in this script.
Python 3, spectra
Run through a python interpreter without any parameter:
python /path/to_this_script/get_simple_gamut.py To pipe the result to a file, do this:
python /path/to_this_script/get_simple_gamut.py > HSL_simplified_gamut.hexplt Identifies the process associated with a window on MacOS. RE: https://superuser.com/a/1171739
Run this script through a Python interpereter withuot any parameter:
python path/to/this/script/macFindProcess.py -- then follow on-screen prompts.
pyenv (suggested) and Python 2 (or will this work with Python 3?), via these commands:
pyenv install 2.7.15
pyenv active 2.7.15 quartz, via this command:
pip install pyobjc-framework-Quartz Renders a PNG image like a single colored bacteria which walks randomly over a surface and randomly mutates (and paints) color as it goes. This is a snapshot of colorGrowth.py which I've decided deserves its own variant script named this. Output file names are random. Inspired and drastically evolved from colorFibers.py, which was horked and adapted from https://scipython.com/blog/computer-generated-contemporary-art/
python 3 with the various modules installed that you see in the import statements here near the start of this script.
Run this script through a Python interpreter without any parameters, and it will use a default set of parameters:
python /path/to/this/script/color_wander.py To see available parameters, run this script with the -h switch:
python /path/to/this/script/color_wander.py -h This is saved as a new script from a development version of colorGrowth.py which has unintended functionality, which is that colors, soon after they have already been filled, still spawn new coordinates and fill them with mutated color for a while. With the -a 1 switch passed to the script and saved animation frames strung together into a video render (e.g. via ffmpegAnim.sh), this appears like ink flowing after it has been poured. This is however much more computationally expensive to simulate, so I saved this development glitch script separate (not intended to be in the final). Renders a PNG image like colored, evolved bacteria (they produce different colors as they evolve) grown randomly over a surface. Output file names are based on the date and add random characters. Inspired and drastically evolved from colorFibers.py, which was horked and adapted from https://scipython.com/blog/computer-generated-contemporary-art/
python 3 with the various modules installed that you see in the import statements here near the start of this script.
Run this script through a Python interpreter without any parameters, and it will use a default set of parameters:
python /path/to/this/script/color_growth_inky_flow_glitch.py To see available parameters, run this script with the -h switch:
python /path/to/this/script/color_growth_inky_flow_glitch.py The content of this file was pasted over color_growth.py; SEE THAT. This file is where an improved version of color_growth.py was forked to (by scribblemaniac), but that improvement was pasted over the original. This file is only here because web links may point to this, and I don't want anyone to hit a dead end looking for it.
Renders a PNG image like bacteria that mutate color as they spread. TRY IT. The output is awesome.
python 3 with numpy, queue, and pyimage modules installed (and others--see the import statements).
Run this script through a Python interpreter without any parameters, and it will use a default set of parameters:
python /path/to_this_script/color_growth.py To see available parameters, run this script with the --help switch:
python /path/to_this_script/ --help GitHub user scribblemaniac sped up this script (with a submitted pull request) by orders of magnitute vs. an earlier version of the script. An image that took seven minutes to render took 5 seconds after speedup.
Output file names are based on the date and time and random characters. Inspired and drastically evolved from color_fibers.py, which was horked and adapted from:
https://scipython.com/blog/computer-generated-contemporary-art
See help for --RANDOM_SEED.
Launches python IDLE via idlelib.
Run through a Python interpreter without any parameter to the script:
python /path/to_this_script/pyenv-idle.py Or, depending on your system setup and if this script is in your PATH, you may be able to run it with just:
pyenv-idle.py Renders a PNG image like colored horizontal plasma fibers via python's numpy and PIL modules. Output file names are random. Horked and adapted from https://scipython.com/blog/computer-generated-contemporary-art/
python 3 with numpy and PIL modules.
Run through a Python interpreter without any parameters:
python path/to/this/script/color_fibers.py Generates a random hex color scheme of file format .hexplt, which is a plain text file with one hex color per line. The generated colors are constructed according to a color theory given in Itten's "ELEMENTS OF COLOR," which states that color combinations tend to be more pleasing to the human eye if, when the colors are mixed (by substractive color mixing), they make gray. This script makes colors that are mixed by additive light (RGB), but the principle is the same and the results have generally verified the theory. EDIT: except not. The way this script is constructed you won't generally ever get brilliant intensity colors by default. I'm not sure anymore things would combine to a tint of gray. You'll almost always want to further modify any resulting palettes that you like; they would be a basis for further work. This is a creative brainstorming tool at best.
Run this script through a Python interpreter, with the --help parameter or examine the parser = argparse code in this script:
python path/to_this/script/NrandomHexColorSchemesGrayMath.py --help -- and examine the help print. Or examine the help= strings in the source code.
Adds the path of whichever directory you call this script from to the PATH (temporarily, or for as long as you have the terminal open) as Python sees it.
Run through a Python interpreter, without any parameter:
python /path/to_this_script/addCurrentPathToSyspath.py Collates documentation comments from all code/script files in the current directory and subdirectories into one file: <repoName>_DOCUMENTATION.md (assumes it is part of a git repository and names the first part of the file after the repository; if that isn't the case it may just end up named _DOCUMENTATION.md), for easier documentation reference. Uses a subset of Mardkwon to allow source code comments loosely styled after Markdown to be converted for final rendered documentation. See USAGE.
In the source code of every script in this repository (or any other repository that you want to use this script for), follow this documentation convention:
REQUIREMENT. Write a documentationHeader.md file which introduces and generally describes this document, or otherwise introduces it. Also a documentationFooter.md document, which has whatever information you want to follow the general body document (which is made from documentation comments in so many scripts) with. NOTE: this script inserts a newline and a "Generated at
REQUIREMENT. Everything before a code begin comment marker is documentation that will be collated into
REQUIREMENT. The code begin comment marker is simply the word 'CODE' (without the quote marks, and with no punctuation and no other words or anything else other than the comment marker, thought whitespace is allowed before or after the comment marker and/or the word 'CODE'). Supported comment markers are: #, //, %, ::, REM, ;, ;;, and possibly others I may add (check the awk command in this script). An example of a code begin marker for a powershell file would be:
% CODE
REQUIREMENT. Document important general information that is suitable for subheaders by writing only a comment marker, and any whitespace before and after, and an all-caps word, and _no other punctuation or character types. Examples of such headers are '# DESCRIPTION', '// USAGE', ';; NOTES', etc. (without the quote marks for any of those examples.) These will be converted to Markdown 4th-level headers.
REQUIREMENT. If the source code is an executable script (such as a python, bash or Perl script) which accepts CLI parameters, give examples of and explain those parameters. Indent example commands to run the script four spaces after the source code comment marker. Eight spaces if they follow a Markdown list item on the previous line (if the command example is with list items). Re: https://daringfireball.net/projects/markdown/syntax
REQUIREMENT. Any documentation of commands that comes after headers and is intended to be presented as technical information (such as example commands) must first have a header, then at least one line of text immediately after that, then the technical information on the next line. This is a limitation of how I parse documentation, but it's also a preference (for example, to introduce the technical information, however briefly, on the line before it).
RECOMMENDATION. If documentation includes double dashes -- as syntax, put the code containing that syntax on a separate line preceded by quadruple spaces, or else Pandoc will convert it to an em-dash for display, which is not correct. But you may also surround the syntax that is part of the -- with backticks for inline preformatted (technical information/code) Markdown.
RECOMMENDATION. Where it is more useful (including for convenience sake), put markdown backticks around words or phrases inline right in text, even if it leads to syntactical confusion in readers of the comment source code (to wonder why those are there).
All documentation lines will have two spaces appended to the end of the line by this script, to force newlines in Markdown (to preserve intended newline breaks).
Even if a script requires no parameters to run, I prefer to give an example command to run the script, under a USAGE header, like this:
scriptName.sh parameterOne parameterTwo
WITH ALL THOSE documentation requirements in place for every source code etc. file in a given repository, and to use this script to make good of so much documentation, and from a directory containing those code etc. files (and this script will also include files in subdirectories), run this script without any parameters:
makeDocumentation.sh (For the _ebDev repo, I run it from within the /scripts folder.)
After parsing and collation, results will appear in <repoName>_Documentation.md.
For the curiosity of listing files newest first according to when they were first committed to git, before you run makeDocumentation.sh, first run this script, from the repository root:
setFileTimestampsToEarliestGitCommit.sh
The resultant <repoName>_Documentation.md file is expected to be large and frequently changing, so good practice is to not store it in a git repository; to exclude it via .gitignore.
Prints the hex values of the data in file $1, with no spaces or newlines or any other information other than the binary digits. Optionally breaks into newlines on every Nth ($2) hex value.
xxd, bash
Run with these parameters:
Example that prints the data of file craft_paper_00006_src.png as hex characters with no newlines or anything else:
printHex.sh craft_paper_00006_src.png Example that does the same but prints a newline for every 2nd character (byte) :
printHex.sh craft_paper_00006_src.png 2 For very large files, splitting newlines over a small number of bytes can be very slow.
To create an array of bytes split on every 8 bytes of hex values from a source file, call this script like this; eightBytesArray would be the resulting array:
eightBytesArray=($(printHex.sh craft_paper_00006_src.png 4))
If you want to rejoin all data from that array (perhaps after manipulating it, for example for data bending or glitching purposes) into a string with no other data, for example as a data basis to dump back to a binary file, you can accomplish that this way, with a here-string and command substitution assigning to a variable:
longDataString=$(tr -d ' ' <<< ${eightBytesArray[@]})
Prints the binary values of the data in file $1, with no spaces or newlines or any other information other than the binary digits. Optionally breaks into newlines on every Nth ($2) binary digit.
xxd Run with these parameters:
Example that prints the data of file inputFile.dat as binary digits with no newlines or anything else:
printBinary.sh inputFile.dat Example that does the same but prints a newline for every 8th bit (or in other words it splits into newlines on every byte) :
printBinary.sh inputFile.dat 8 Like many things in bash/Unix, you can pipe the result to a file:
printBinary.sh inputFile.dat 8 > inputFileBinaryValues.txt
For very large files, splitting newlines over a small number of bits can be very slow.
To create an array of bits split on every 8 bytes of binary values from a source file, call this script like this; bytesArray would be the resulting array:
bytesArray=($(printBinary.sh inputFile.dat 8))
If you want to rejoin all data from that array (perhaps after manipulating it) into a string with no other data, you can accomplish that this way, with a here-string and command substitution assigning to a variable:
longDataString=$(tr -d ' ' <<< ${bytesArray[@]})
Calls dcraw2tif.sh for every file of type $1 (should be a camera raw format file) in the current directory, and optionally all subdirectories.
Run with these parameters:
Example that will convert all cr2 files in the current directory to tiff format files:
dcrawAllType2tif.sh cr2 Example that will do the same for all cr2 files in the current directory and all subdirectories:
dcrawAllType2tif.sh cr2 FROBYARF Copies metadata from file $1 to $2, if possible, via exittool. Any metadata fields that already exist in the target are overwritten with metadata from the same tags in source. No backup of the target file is made; the change is permanent and, if there are errors, destructive.
exiftool, a nixy/bash environment to run this script (e.g. MSYS2 on Windows)
Run with these parameters:
Example that will copy metadata fom file source.mov to target.mp4:
copyMetadataFromSourceFileToTarget.sh source.mov target.mp4 Example that will do the same but also update the time stamp of target.mp4 to the media creation date and time from metadata:
copyMetadataFromSourceFileToTarget.sh source.mov target.mp4 FNEORN Inclusion text filter. Prints text file $2, modified, with only words that are also in text file $1. Another way of saying that is: prints only words in $2 that appear in $1. Both files must have one word per line.
Run with these parameters:
For example:
filterIncludedWords.sh incudeWords.txt textToFilter.txt The source text files are expected to have one word per line. This will not work the way you might want with paragraphs.
To capture results to a new file, use a redirect operator, like this:
filterIncludedWords.sh incudeWords.txt textToFilter.txt > filteredWords.txt
For preparation for data bending animation. Could have other purposes I don't know. Takes source file $1 and copies increasing numbers of bytes from it to numbered files named after it, in a subfolder also named after it (plus some random characters). Bytes are in multiples of optional parameter $2 (default value 1). Also, file names of byte copies are given the extension .dat.
Run with these parameters:
For example:
copyIncrementedBytes.sh A_Screed_Into_the_Void.txt To use this from another script and make use of the subfolder name, call it with source:
source copyIncrementedBytes.sh A_Screed_Into_the_Void.txt -- and then use the $copyIncrementedBytesSubfolderName, which will be set in your environment if you call this script and return from it that way.
print, copy, byte, data bending, data bent, animation, type, increment
Runs extractXMPandEmbeddedRAWfromDNGandDestroyDNG.sh for every DNG format file in the current directory.
Run without any parameters:
extractXMPsandEmbeddedRAWsfromDNGsandDestroyDNGs.sh Extracts embedded raw image and xmp sidecar from DNG file $1, then deletes the DNG. (This delete is permanent and irreversible!) Does not overwrite raw image files that would be overwritten if they already exist, and notifies that they already exist. Also, warns if extracted raw file is different from any found raw file of the same file name. Also, it sets the timestamps of the extracted raw files (from their metadata) to match their creation/modify dates.
adobeDNGconverter, exiftool, toOldestWindowsDateTime.sh, and binarez_touch (Windows only!), all in your PATH.
Run with these parameters:
For example:
extractXMPandEmbeddedRAWfromDNGandDestroyDNG.sh DSC_0150.dng How this avoids clobbering raw files that already exist:
Sorts (filters) all images in the current directory which have similarity threshold $1 to image $2. Moves those sorted images into a randomly named subfolder (after the pattern similar_images
printAllIMGfileNames.sh, GraphicsMagick, image files in the current directory to work on, and bash / GNU utilities
Run with these parameters:
runs rgbHexColorSortInOkLab.js against all .hexplt files in the current directory (and optionally all subdirectories), overwriting the original files with the result.
getFullPathToFile.sh, rgbHexColorSortInOkLab.js.
With more than one .hexplt file in your current directory, and rgbHexColorSortInOkLab.js in your PATH, run with these parameters:
rgbHexColorSortInOkLab.js to sort perceptually by next nearest color to this color). If provided, it must not have the '#' symbol at the start of the sRGB color (for example, to use a magenta color, just pass f800fc, not #f800fc). If not provided, no arbitrary first color parameter will be passed to rgbHexColorSortInOkLab.js..hexplt files in all subdirectories under the current directory, and call rgbHexColorSortInOkLab.js for all such found .hexplt files (in subdirectories) also. If omitted, only .hexplt files in the current directory will be found and passed to rgbHexColorSortInOkLab.js. To use this paramter but not $1, pass the word NULL for $1.Example that will call rgbHexColorSortInOkLab.js repeatedly for every .hexplt format file in the current directory, with no additional parameter:
allRGBhexColorSortInOkLab.sh Example that will call rgbHexColorSortInOkLab.js repeatedly for every .hexplt format file in the current directory, with a parameter telling it to set #0a000a (a magenta black) as the arbitrary first color to start comparisons with for every .hexplt file:
allRGBhexColorSortInOkLab.sh 0a000a Example that will call rgbHexColorSortInOkLab.js for every .hexplt file found in all subdirectories also, but not specify any arbitrary first comparison color:
allRGBhexColorSortInOkLab.sh NULL FROGBALF Example that will specify an arbitrary first comparison color and operate on all .hexplt files in all subdirectories also:
allRGBhexColorSortInOkLab.sh f800fc FROGBALF SEE ALSO allRGBhexColorSortInCAM16-UCS.sh and allRGBhexColorSortInCIECAM02.sh.
runs reformatHexPalette.sh with the same parameters for every .hexplt file in the current directory. Uses same parameters as that script (SEE) but shifted. See USAGE.
reformatHexPalette.sh in your PATH.
Call this script with these parameters:
Example that passes 16 for the columns parameter:
reformatAllHexPalettes.sh 16 Example that passes 16 for the columns parameter and 8 for the rows parameter:
reformatAllHexPalettes.sh 16 8 This script provides $1 as a different file name (pulling from every .hexplt format file in the current directory), for every call it makes to reformatHexPalette.sh. What is provided as $1 for this script is passed as $2 for that script, and $2 is passed for $3.
concatenates all .hexplt files in the current directory into one, then deduplicates colors from it and sorts it in okLab color space (using various scripts).
nodejs, the culori package for it, and various scripts in _ebDev
Run without any parameters:
joinAllHexpltsAndSortInOkLab.sh Result palette name will be named after the current directory plus random characters, and printed for your information after operations are complete.
Reformats a .hexplt file (a list of sRGB colors in hex format) to remove all comments and arrange colors on an $1 column by $2 rows grid, then add back a comment that tells the grid dimension (appended to the first row).
Run with these parameters:
For example, to reformat a .hexplt file with defaults, run:
reformatHexPalette.sh hobby_art_0001-0003.hexplt To reformat the same with
Assists in image organization and color sampling via get_color_sample_grids_sRGB.sh. Organizes all images (in the current directory) into subfolders named after the dimensions of the sample grid for which colors will be sampled from them, then repeatedly calls get_color_sample_grids_sRGB.sh with parameters appropriate to those image collections per folder. Presents all images in the current directory one by one and prompts user for the column and row dimensions to sample for each image respectively, then sorts into subfolders to that end (and subsequently samples all images in all subfolders) accordingly.
printAllIMGfileNames.sh, get_color_sample_grids_sRGB.sh, and their dependencies, irfanView and/or any other default image viewer, and an environment that will open an image to the default image viewer using the start command.
Alter the hard-coded x offset and y offset variables (xSampleOffset and ySampleOffset at the start of the script (right after the CODE comment) per your want. (See "$4..X percent offset to sample from left edge of cells.." and "$5..Y percent offset.." parameters in get_color_sample_grids_sRGB.sh (which in turn map to the Python script that calls).
Then, run this script without any parameters:
get_color_sample_grids_sRGB_assistant.sh -- and follow the prompts.
Calls augmentPalette.sh for every .hexplt file in the current directory, with a new file but the same augment (interpolation) parameter for every call. Writes each result to a new file named
Run with these parameters:
For example:
augmentAndRenderAllPalettes.sh 32 Does linear interpolation in N ($2) steps between each color in a .hexplt palette ($1), in okLab color space (using CHL coordinates). Prints the result to stdout.
get_color_gradient_OKLAB.js (and nodeJS and the packages that script requires), getFullPathToFile.sh
Run with these parameters:
For example, to interpolate 13 colors in between each color in the file Firebird.hexplt, run:
augmentPalette.sh Firebird.hexplt 13 The result file for this example would be Firebird_augmented_13.hexplt.
This script checks for a local environment variable $fullPathToOKLABAugmentationScript, which is the full path to the dependency script get_color_gradient_OKLAB.js. If that environment variable is not set, it will look for the path to the dependency script via getFullPathToFile.sh, and set that variable to what getFullPathToFile.sh finds. If you call this script via source, that newly set variable will still exist in your shell when this script returns. This will save a call to getFullPathToFile.sh if you call this script (augmentPalette.sh) again, which can save a lot of time, as that path finding script can run very slow. (It saves time because after the first call, the variable is set and so on the second call, it checks and sees it's already set and doesn't look for it again.) So, if you call this script repeatedly from another script, call it with source this way, where <palette_file_name> and/or <N> change with each call:
source augmentPalette.sh
To write the printed result to a file, redirect it like so:
augmentPalette.sh Firebird.hexplt 13 > Firebird_augmented_13.hexplt
Finds all empty directories in the current directory and their subdirectories (if they exist) and deletes them. Does so recursively; also removes empty directories that result from removal of an empty subdirectory.
I don't know how this script might misbehave on directories with hidden files etc. Use at your own risk.
Run without any parameters:
deleteEmptyDirectoryTrees.sh It prints feedback of anything it deletes as it runs. If it prints nothing, no empty directory was found.
Last of a series of scripts designed to identify similar .hexplt format palettes. Takes palettes discovered by listPaletteDifferencesBelowThreshold.sh (which must be run first), which are similar, and copies them into new subfolders named after the palette they are similar to. Also copies any accompanying .png palette render images into the same folders. Can alternately move hexplt and palette images (instead of copy). For purposes of anything you would want to do with palettes that can be seen as perceptually similar, for example eliminating palettes so perceptually similar as to be practical duplicates, or grouping similar palettes.
Before this script, run listPaletteDifferencesBelowThreshold.sh (which tells you to run another script before it), to generate the list paletteDifferencesBelowThreshold.txt. Then run with these parameters:
For example, to copy all found similar palettes into subfodlers named after pair A for each discovered similairity, run:
groupPalettesDifferentBelowThreshold.sh To move palettes into subfolders instead of copying them (with possible side effects detailed under NOTE), run:
groupPalettesDifferentBelowThreshold.sh FROGBALF It will create a new subfolder for any palette for which one or more similar palettes were found, and copy that palette with the palette(s) similar to it into that new subfolder. You may then examine the resulting subfolder(s) for whatever further creative purposes you have (like deleting similar ones you like less, or copying them to a group of palettes elsewhere, or whatever).
If you use parameter 1 (opt to move files), you may end up with folders where an A or B similarity pair wasn't moved, and empty subfolders. That may mean that one palette (B) had similarity to two or more other palettes (A and C [D, E..]) which were not similar enough to each other for grouping; e.g. B is within similarity threshold to both A and C, but A is not within that similarity threshold to C.
Dangerous deduplication. Lists all files in the current directory (or optionally reads from a file list), then moves all copies of those files that have the same file name, which Everything search engine (voidtools) finds, everywhere else on the computer, into the current directory. A potentially false assumption here is that the other files are the same file. If you know that they are not the same; if they are changed; if you copied them elsewhere and modified them, and want to move them back, this would be an advanced "move these changed files back" auto-thingie. You would probably only want to use this script if moving files back becomes complicated; for example if you split them into various revision/sorting folders before changing the copies. USE WITH EXTREME CAUTION, knucklehead.
Moves and overwrites from this script are permanent and immediate, but it prompts to type a confirmation password first. See details in the "echo" code statement in this script.
Run with these parameters:
$1 OPTIONAL. File name of a text file in the current directory which lists files to be moved here from everywhere else that Everything CLI (es.exe) finds them. Only files listed in this text file will be so moved. For example, if you have a file named filesToMoveHere.txt, with contents like this:
Hfwuzjuz.hexplt
Hfwuzjuz.png
RSImFGzg.hexplt
RSImFGzg.png
SPhNATb4.hexplt
SPhNATb4.png
YWgZmFP3.hexplt
YWgZmFP3.png
fu5jGjvP.hexplt
fu5jGjvP.png
-- then all copies of those files (from the list) which Everything CLI finds elsewhere on the computer will be moved into this directory (whichever directory your terminal is in when you call this script), each one overwriting the last until the last is moved.
Example that will do naive deduplication of every file found in the current directory:
lsEverythingMoveHere.sh Without any paramter, that way, all files in the the current directory will be listed and all copies of them found from everywhere else on the computer will be moved to this directory.
Example that will do naive deduplication of all files from a file list:
lsEverythingMoveHere.sh filesToMoveHere.txt Example that will recurse through all subdirectories and move all files into that directory (which are found elsewhere on the computer that have the same file name as files in that directory):
lsEverythingMoveHere.sh NULL YORFPLERION Lists all files in the current directory, then deletes all copies of those files that have the same base file name (meaning no extension), which Everything search engine (voidtools) finds everywhere on the computer. THAT MEANS all of the files found in this folder, too. USE WITH EXTREME CAUTION, knucklehead.
Deletes from this script are permanent and immediate, and all files in the current folder you run this from are part of that deletion. It prompts to type a confirmation password first.
lsEverythingDelete.sh calls SVGrandomColorReplace.sh $1 times with additional parameters to be passed on to that script (see USAGE), but creating copies of the source svg (named after pattern Y_m_d__H_M_S__N), and in a subfolder named after the original file plus random characters) before calling that script to modify the copies.
Run with these parameters:
Example that will create 38 copies of 2021-09-13-zb_v5.svg with random color replacements via optional parameter $3 (a source .hexplt file, earth_pigments_dark.hexplt), replacing hex color 000000 via optional parameter $4:
SVGrandomColorReplaceCopies.sh 2021-09-13-zb_v5.svg 38 earth_pigments_dark.hexplt 000000 If you call this script from another script a certain way (which, I know, this script calls another script) in turn), the $subDirForRenders variable which this script sets will be available (as a new global or environment variable) in the shell that called this script. The way to set that variable in the calling environment is to use source before calling this script, like this:
source SVGrandomColorReplaceCopies.sh <script parameters> -- which may be handy for use of this script in a "recipe" to do multiple passes of replacing specific colors with random selections from palettes. Handy, because if you know the name of the random subfolder it made, you can cd into it to list svg files into an array and modify the files with additional passes.
Fast global search within Windows files. Gets a list of all file names that match search string $1 (via voidtools Everything), then prints any results (with the file name) of grep search $2 from them. Windows only, so far as I know. SEE ALSO: advanced search menu in Everything, and restricting search to a specific folder via Everything right-click shell menu option.
Run with these parameters:
For example:
everythingGrep.sh .sh BWsvgRandomColorFill for advanced searching, refer to https://www.voidtools.com/support/everything/searching/ (for example, to find files that end with a certain string, search for:
endwith:.sh -- but you may need to surround your search with single quote marks if it contains unusual characters, like this:
everythingGrep.sh 'endwith:.sh' BWsvgRandomColorFill -- or also (per DESCRIPTION) see advanced search options menu in Everything.
Repeatedly calls copyVidLosslessEncodeSound.sh with every file of type $1 in the current directory as a parameter for each call. (Converts so many video files of type $1, using that script.)
ffmpeg, copyVidLosslessEncodeSound.sh Run with these parameters:
For example, to convert every .MOV file in the current directory via copyVidLosslessEncodeSound.sh, run:
copyVidLosslessEncodeSoundAllType.sh MOV Copies the video stream of a source video into a new container and re-encodes the audio in aac high quality, into an mp4 container. Intended for slightly modified video distribution from source video from some (silly) devices that record AVC/PCM videos (which have like 1:10 ratio waste) in sound stream size via PCM; ergo this script to get them compressed audio to save space. Output files are named after the input file, but add ~_aacSound to the file name. Also, metadata and timestamps are copied from the source to the target via another script.
ffmpeg, copyMetadataFromSourceFileToTarget.sh. Run with these parameters:
For example:
copyVidLosslessEncodeSound.sh inputVideo.MOV creates a 1200x1200 png resized copy, by nearest neighbor method, of every png in the current directory. Intended for square color field previews of palettes rendered via renderAllHexPalettes.sh. Files are named after the original but append 1200x1200nn to the base file name.
This overwrites any target file names of the same name without warning.
Run without any parameters:
allPNGto1200x1200nn.sh Runs getUniqueWords.sh for all files of type $1, OVERWRITING the originals with the found list of unique words (in original order).
Run with these parameters:
For example:
getUniqueWordsForAllType.sh hexplt Creates tweaked copies of all hexplt files in the current directory, with the possible changes including chroma (c), and/or hue (h), and/or lightness (l). Accomplishes this via repeat calls and capture of output from print_altered_hexplt_OKLAB.js. Modded hexplt files are named after the original, but adding _mod to the file name.
If you already have hexplt files modified from originals but with _mod.hexplt in the file name, this script will clobber (overwrite) them without warning.
A bash or bash-like environment, nodejs, and print_altered_hexplt_OKLAB.js and the node librarires it requires.
Run with these parameters:
For example:
alterAllHexpltsOKLAB.sh '-c 0.018 -l 0.068' Calls get_color_sample_grid_sRGB.py for every image file of type $1 (optionally: all supported image types) in the current directory. Passes to that python script parameters of the same type and position as that script supports, with an option to automatically calculate number of rows to match column width (square sample cells). Captures the outputs of the python script and writes them to .hexplt files named after each source file.
python, get_color_sample_grid_sRGB.py and the Python librar(ies) it requires, and, getFullPathToFile.sh, printAllIMGfileNames.sh.
Run with these parameters:
--Whew!
Example command that will operate on every png file in the current directory, sampling 16 columns for each, with an automatically calculated number of rows to :
get_color_sample_grids_sRGB.sh png 16 Example that will sample 16 rows and 2 columns for every png in this directory:
get_color_sample_grids_sRGB.sh png 16 2 Sample 16 rows and 2 columns for every png in this directory, and offest the sample at thirteen percent (0.13) from the left edge of each cell:
Sample 16 rows, 2 columns, from every png in this directory, offest the sample at thirteen percent (0.13) from the left edge of each cell, and twenty percent (0.2) from the top edge of each cell:
get_color_sample_grids_sRGB.sh png 16 2 0.13 0.2 Do all of that except use the defautl offsets, and sample colors from all png images in all subdirectories:
get_color_sample_grids_sRGB.sh png 16 2 DEFAULT DEFAULT WHEALHALM Do all of that but automatically calculate the number of rows to sample so that sample cells are square:
get_color_sample_grids_sRGB.sh png 16 AUTO DEFAULT DEFAULT WHEALHALM Alternately sample all file types:
get_color_sample_grids_sRGB.sh ALL 16 AUTO DEFAULT DEFAULT WHEALHALM -- Double whew!
Animate RND block noise as in makeBWGridRandomNoiseAnim.sh, and use it as alpha in a composite animation with a foreground image animated over a background image, where the RND block noise is the animated transparency (or alpha) mask. Uses complete foreground image as first still and background as final still (with RND block noise anim between).
ffmpeg, graphicsmagick (as gm), imgs2imgsNN.sh, ffmpegAnim.sh.
Run this script with these parameters:
bg.png if not provided.fg.png if not provided.Example that will generate images that are 5 columns wide, 8 rows high, make 28 such images, animate them at a source framerate of 0.65 per second, and blow them up to 746 x 1080px, assuming the image files bg.png and fg.png are in this directory:
makeBWGridRandomNoiseAlphaComposite.sh 5 8 28 0.65 746 1080 Organizes exported media files (e.g. from cameras and devices), using other scripts, this way:
renames various common camera/device media file extensions to lowercase
renames files after metadata date
renames all .jpeg file extensions to .jpg (including doing this after optional thumbnail extraction, where dcraw extracts thumbnails as .jpeg files)
For the following optional features, search for comments with "uncomment" instructions:
extracts embedded thumbnails from cr2 raw files. (Performs the rename of .jpeg files .jpg after this.)
losslessly copies all video files to mp4 containers, with metadata and timestamps copied from the original.
moves all relevant data into subfolders named by type.
Various scripts. See throughout the code.
From a folder with such files to organize (and no other files!), run without any parameters:
organizeCameraExports.sh See the various comments with "uncomment" instructions for the optional features.
dcraw, a bash environment that can run this script (e.g. what comes with a Mac, or MSYS2 on Windows), toLowercaseExtensions.sh
Run with these parameters:
Example command to operate on all NEF format files:
getEmbeddedThumbsDCRAW.sh NEF Example command to operate on all CR2 format files in the current directory and all subdirectories:
getEmbeddedThumbsDCRAW.sh CR2 foo DANGEROUS. Renames all files and folders found on a system such that a search string is substituted with a replace string, WITHOUT PROMPT OR WARNING, but requiring a password parameter. Logs renames to a text file. See "DON'T DO THIS UNLESS" under USAGE.
THIS CAN BREAK STUFF HARD AND FAST AND PERMANENTLY IF MISUSED. See "DON'T DO THIS UNLESS" section under "USAGE."
Voidtools "Everything" search engine, working and showing files you search for correctly, and accompanying es (CLI tool for it) in you PATH, probably MSYS2 bash environment.
Run with these parameters:
Example that will replace the string "dGSUyfhH" with "Murky_Forest" in all files:
everythingRename.sh dGSUyfhH Murky_Forest YOINK Example that will delete the string SNAIRFU from all found files:
everythingRename.sh SNAIRFU -_-SNIP-_- YOINK DON'T DO THIS UNLESS:
Renames all folders in the current folder (directory) by unix date stamp to custom format, based on the newest file found inside each folder. Not recursive: subfolders of folders are not found or operated on. Prompts to confirm operation.
Run without any parameter, and follow the prompt:
renameFoldersByDate.sh To override the prompt and force renames, pass one parameter, which is the word HARCHOOF:
renameFoldersByDate.sh HARCHOOF Fetches N random images from earthview.withgoogle.com and runs diff_avg_supercomposites.sh.
Run with one parameter, which is how many random images you want to retrieve before this script calls diff_avg_supercomposites_nested_loop.sh; for example:
rnd_withgoogle_earth_view_diff_avg_supercomposites.sh 12 diff_avg_supercomposites_nested_loop.sh after image colleciton instead, comment out the line for .sh, and uncomment the line for the other option.Renames all extensions of all files of type $1 in the current directory, which have any uppercase letters in them (such as PNG), to lowercase (png). Optionally does this to all files of every type, also optionally to all subdirectories of the current folder.
Run with these parameters:
To rename all PNG files in the current directory which have have uppercase letters in their extensions to all lowercase in their extensions, run:
toLowercaseExtensions.sh png To rename all files with every extension that has uppercase letters in them to have all lowercase letters in their extensions, run:
toLowercaseExtensions.sh ALL_EXTENSIONS To rename all MOV files in the current directory and all subdirectories which have any uppercase letters in their extension so that they are all lowercase in the extension, run:
toLowercaseExtensions.sh mov foo To rename all extensions with uppercase letters in all file types in the current directory and all subdirectories, run:
toLowercaseExtensions.sh mov ALL_EXTENSIONS foo Makes a (non-standard?) ppm image which is checkers of color $1 and $2, at dimensions $3 x 2 by $4 x 2. The checkers are 1 pixel high and wide. Create a large image of this and squint to optically blur it, and you've got optical blending of two colors. Useful to figure out how any two colors perceptually blend (for color matching in gradients or for other creative/artistic purposes). Target file is named after source colors and dimensions. Will clobber existing files.
This script clobbers (overwrites) existing target files without warning.
Run with these parameters:
Example command that will create a checker ppm of sRGB hex color #ff9710 (an orange) and #feff06 (a yellow) 2048 pixels across and 2048 pixels down, because it the third parameter is the number of times to repeat the color pair (so $3 x 2):
makeOpticalBlendingCheckersPPM.sh '#ff9710' '#feff06' 1024 2048 See img2imgNN.sh to convert the resulting ppm to png.
Creates a still color video the duration of a source sound, in an mp4 media container, copying the sound losslesly into the video container. Useful for sharing music on silly platforms that only share video but not sound.
randomString.sh, graphicsmagick (convert), ffmpeg
Run with these parameters:
Example that will create a video of default size and color from one input sound file:
addColorStillVideoToSound.sh Final_Fantasy_Legends__track_01.mp3 Example that will create a video of size 720x720 of default color from one input sound file:
addColorStillVideoToSound.sh Final_Fantasy_Legends__track_01.mp3 720x720 Example that will create a video of size 720x720 with red still color in hex format:
addColorStillVideoToSound.sh Final_Fantasy_Legends__track_01.mp3 720x720 '#ea0000' This clobbers (overwrites) the target video file if it already exists, without warning.
Deletes (rm) all files of type $1 in range $2 to $3 (N1 to N2), in the current directory. Prompts to type a displayed password to confirm you wish to do so. Sorts by default of find command.
Run with these parameters:
To delete all files of type png which are listed by the find command from the 51st to the last found png:
rmn.sh png 51 To delete all files of type png from the 1st to the 29th found:
rmn.sh png 1 29 Runs a series of scripts to prepare and embed metadata in final exported (pre-publication) art etc. files.
Not currently well documented. A start at documentation:
Calls get_color_gradient_OKLAB.js repeatedly to construct palettes from files in a source .hexplt file ($1), such that:
In other words, this script obtains and lists a gamut of tints, shades and saturated and unsaturated colors for every color in a palette. Results will be in palette files named after each color in the source palette. Also, this calls a script to render the result palettes in a layout that shows the tint, shade and chroma gradients' relationships.
DEPENDENCIES:
getFullPathToFile.sh, get_color_gradient_OKLAB.js
Run with these parameters:
get_color_gradient_OKLAB.js. If there are spaces in these switches, surround all the switches and their values with single or double quote marks. To not use any arbitrary (additional) switches, but use $7, pass the word NULL for this parameter.Example that will get 3 tints, 3 shades, and 4 chromacities for every tint and shade, for every color in 16_max_chroma_med_light_hues.hexplt:
getN_hexplt_shadesXchromas_Oklab.sh 16_max_chroma_med_light_hues.hexplt 3 4 extraParams variable to pass additional arguments in calls to get_color_gradient_OKLAB.jsNOTE THAT get_color_gradient_OKLAB.js is recommended over this script for color gradients. This script, via a combination of scripts (which in turn call scripts), gets M shades times N chromacities (color saturation intensities) of every color in a palette. Essentially, obtains and lists a gamut of tints, shades and saturated and unsaturated colors for a palette. Results will be in palette files named after each color.
Run with these parameters:
Example that will get 7 shades and 6 chromacities for every resultant shade, for every color in 16_max_chroma_med_light_hues.hexplt:
getNshadesXchromacities_from_hexplt.sh 16_max_chroma_med_light_hues.hexplt 7 6 Again, `get_color_gradient_OKLAB.js' is recommended over this script for color gradients.
For all files of a given type (parameter $1) in the current directory (and optionally all subdirectories), moves them into a new subdirectory (in the immediate folder) named after the base file names. Creates that subfolder only if it does not already exist, and moves files to them only if they do not already exist. SEE ALSO moveToTypeFolder.sh.
Run with these parameters:
Example that will sort all files with the extension .hexplt into new subfolders named after the files:
moveTypeToBasenamedFolder.sh hexplt Example that will sort all files with the extension .hexplt in the current folder and all subfolders into a new directory named /hexplt:
moveTypeToBasenamedFolder.sh hexplt EKTHELPOI Converts an .rgbplt format palette (a list of RGB values, in decimal) to a list of RGB colors in hex format. This script adapted from hexplt2rgbplt.sh (see also).
Run this script with one parameter, which is the .rgbplt format file to convert, e.g.:
hexplt2RGBplt.sh RAHfavoriteColorsHex.rgbplt If you have a file ~/palettesRootDir.txt with a root path to search for .hexplt files in it, this script will search all paths below that root folder for the file, IF the file is not in the same directory you run this script from. If the file is in the same directory, it uses it from the same directory.
Alters any vector file that InkScape can edit to have the viewable area set to the drawing, with no padding. Not very efficient (load, operation and save is slow per file), but the only way I've found at the moment to do this.
inkscape installed and in your PATH.
Run with these parameters:
For example:
inkscapeResizeCanvasToDrawing.sh svg Calls ffmpeg2imgNN.sh repeatedly for every file of type $1, converting to format $2, upscaled to target max dimension $3.
Run with these parameters:
Example that will convert all png images to bmps scaled up with 1280 pixels on their longest side:
ffmpeg2imgsNN.sh png bmp 1280 Via GraphicsMagic, overwrites every image of format $1 in the current directory with a new image which is the same size (dimensions), of flat color $2. See USAGE. As this is a permanently destructive (or transformative) action, the script requires the user to type two different given confirmation words to do this.
Run with these parameters:
Example that will overwrite all png images in the current directory with blank images of a default color:
blankAllimgsType.sh png Example that will overwrite all png images in the current directory with what red as defined by GraphicsMagick:
blankAllimgsType.sh png red Calls extractIMGlayersIM.sh repeatedly for every file of type $1 (for example psd) in the current directory, and optionally subdirectories. Passes $1 as the first parameter to that script for each call.
Run with these parameters:
extractIMGlayersIM.sh with that file name as the parameter to that script. See that script's documentation for detailsExample command that will call extractIMGlayersIM.sh once with every psd (Photoshop format) file in the current directory:
extractIMGsLayersIM.sh psd Example command that will do the same but for all psd files in the current directory and subdirectories also:
extractIMGsLayersIM.sh psd FLOREFLEFL Retrieves the aspect of an image as a decimal and informs you (to do: and does stuff based on) whether the image falls within Instagram's allowed aspects of 0.8:1 to 1.9:1 (WxH) for uploads.
NOTE Image aspects and dimensions allowed by Instagram are 0.8:1 through 1.9:1 and it may upload images larger than 1080px on a side but it will always shrink them to that if not to 600 px on a side. re: https://help.instagram.com/1631821640426723 -- and other URLs in comments in this script.
Run with one parameter, being the image file name to check for Instagram aspect restraints fit (for upload), e.g.:
getDoesIMGinstagram.sh ./_EXPORTED_M_variantWork_00099_FFsideToside_v02_PZ-8280x.jpg NOTE that if you run this script with the source command before it, e.g.:
source getDoesIMGinstagram.sh ./_EXPORTED_M_variantWork_00099_FFsideToside_v02_PZ-8280x.jpg -- then the boolean variable $doesInstagram created by this script survives in the shell (for e.g. another script to use) after this script returns.
GraphicsMagick, and a 'nix environment with the bc command-line calculator.
Runs getNshadesOfColorCIECAM02.py N times (per default parameter or $1) to create grayscale palettes which run perceptually darker from N divisions of white to black. For every palette, it also renders a png from the palette on one line to provide a gradient image.
Run without any parameter to use the default number of palettes to create:
create_perceptual_gray_gradient_palettes.sh To create N palettes, pass any integer as the first parameter; for example to create 25 palettes, run:
create_perceptual_gray_gradient_palettes.sh 25 Counting starts at 2, which is just black and white, so that you can stare at black in one eye and white in the other eye via binoculars (and go insane) and see what your brain does with that.
Prints file names and creation dates for every file in the current directory, one file per line, with the file name and date separated by a bar |. If an optional parameter is passed, does this for all files in all subdirectories also.
Run without any parameter:
printFileNamesAndCreationTimes.sh Run with one optional parameter, which may be anything (for example the word 'FLURFBLORG'), to print information for all files in all subdirectories as well:
printFileNamesAndCreationTimes.sh FLURFBLORG Calls two scripts to join ventilated prose in a fountain format file, then render it to PDF named after the original file.
Two other scripts must be in your PATH, and also the dependencies of those scripts. Those scripts are:
fountain2fountain.sh
fountain2pdf.sh
While this script backs the original fountain file, modifies a copy of it, and then restores from backup, if this script is interrupted or anything else goes wrong, your original fountain file may become damaged or lost. Therefore, back up any file you run this against before you run this.
Run with these parameters:
Examples:
fountain2fountain2pdf.sh ascent_to_guru_peak__sps9.fountain
fountain2fountain2pdf.sh ascent_to_guru_peak__sps9.fountain FLOREFLEFL FLOREFLEF This script temporarily swaps the file names of the source fountain file and the unventilated converted fountain file. If you interrupt the run of this script, be aware of the potential wrong file names problem that can result of this script being interrupted before it finishes swapping the file names back. Should that happen, know that the original fountain file is named renamed to "$fountainSourceFileName"_tmp_rename_8qeMAvAyp.fountain, and the unventilated one is renamed to the original fountain file (again, both temporarily, but it may stay that way if you interrupt the script run).
Last of a series of scripts designed to eliminate similar .hexplt format palettes. Interactively launches pairs of rendered palettes corresponding to .hexplt files found to be similar (not so different below a rating threshold), as discovered by listPaletteDifferencesBelowThreshold.sh (which must be run first). Logs how many comparisons have been examined and allows resume from Nth comparison (see USAGE).
For assistance with managing copies of palettes (in favorites collections), this script optionally uses everything CLI (es.exe, if it makes a successful attempt to run an executable of that name in a way that will produce no error), for statistics on how many copies of given images are found on your (Windows) computer. See USAGE for details.
Before this script, run listPaletteDifferencesBelowThreshold.sh (which tells you to run another script before it), to generate the list paletteDifferencesBelowThreshold.txt. Then run with or without this parameter, and follow the prompts:
For example if the log file reads:
Iteration 320 of 794, pair CikkUp54.png | DNRi4NKs.png . . . To resume from that comparision, you may run this script with that first number:
deletePalettesDifferentBelowThreshold.sh 320 The script will skip all iterations up to 320 and resume from there.
To examine all iterations, run this script without any parameter:
deletePalettesDifferentBelowThreshold.sh Allows quick workup of many color pair options for designs. Swaps the first and last color (line) from every .hexplt file in the current directory (optional: also subdirectories) into copies of SVG file $1, named after the SVG and .hexplt file. The files are in a subdirectory named <SVG_file_base_name>_colorOptions.
From a directory with .hexplt files in it, run with:
SVG file name which you want to create SVG variant files with color options for.For example:
BWsvgColorSwaps.sh design.svg Creates a looped video in which all images of type $1 in the current directory are crossfaded one after another, and then the final image crossfades back to the first. Accomplishd by custom logic in this script and runs of ffmpegCrossFadeVideos.sh and concatVideos.sh. Result video renamed after this directory.
Intermediate videos are lossless though compressed (utvideo codec) avis, and they are much larger compared to mp4s. This script does not delete those files, as you may wish to keep any of them for archiving or resource combination purposes. If you don't want those large files, you may want to delete them afterward.
Run with these parameters:
Example that will make a crossfade video from all pngs, with default durations:
ffmpegCrossFadeAllTypeIMGs2video.sh png Example that will make each still image video 4.7 seconds, and crossfade for 2.6 seconds between them:
ffmpegCrossFadeAllTypeIMGs2video.sh png 4.7 2.6 Result file is named after the current directory and source image type, after the pattern __<directoryName>_<$1>_crossfades.mp4.
Finds all files (recursively) with a name containing the same base file name as all files listed in file $1, and copies them up from subfolders into the current directory. Uses include copying preset files that accompany copied (favorite) renders. Will not clobber files in the current directory that already exist.
Run with these parameters:
For example:
copyUpMatchedFilesFromList.sh possible_color_pairs_from_RAH_favorite_colors_2_combos.txt Render so many png images from presets
Open all result renders in a thumbnail image navigator like IrfanView
Copy favorites to the project parent folder via IrfanView functionality that allows that (sorry, not detailing that)
Run a command like this to list all those copied renders to a text file:
ls *.png > favorites.txt
Run the following command (assuming this script is in your PATH) to copy associated render configuration etc. files up from the subfolders:
copyUpMatchedFilesFromList.sh favorites.txt
Sort all those copied files into another folder for further work
YOU MAY ALSO get such a list by piping output of listUnmatchedExtensions.sh to a file, like this:
listUnmatchedExtensions.sh png svg > noMatchingSVGs.sh pair, matched, copy, up, file list
Plays a system "beep" N times (per default or parameter $1), with a pause in between each beep. Useful as a notification after run of a command that takes a long time.
Note that your terminal settings need to have any system beep enabled (for MSYS2 you can find this under the options with a right-click of the terminal menu bar). If that is set, run this script without an optional parameter:
Example that would play 5 beeps (or whatever sound the system makes):
systemBeepXn.sh 5 Example that will play the default number of beeps:
systemBeepXn.sh To run this script after another script (for example as a notification after a long process ends), follow the run command of that script with a double-ampersand && and then the call of this script, for example:
makeDocumentation.sh && systemBeepXn 7 Takes an input sound file $1 and still image $2, and creates a video still from $2 matched to duration $1. Then muxes the sound and video together. Output file is named <$1>_still_$2.mp4.
Run with these parameters:
For example:
stillIMGAndSoundToVid.sh input.wav still.png Creates converted copies of all images of many types in the current directory to format $1.
From a directory with images in it of varying types, run this script with one parameter, which is the target format (or extension, without the . in it) to create converted copies of. For example, to create converted copies of many images in .jpg format, run:
IMGsTypesToIMGs.sh jpg Creates a converted copy of image file name $1 to format $2, via ImageMagick. Will not convert if render target already exists. Optionally downsizes (with good downsizing method) via $3.
ImageMagick installed in your PATH.
Run this script with these parameters::
For example, to create a converted copy of color_growth_title.psd to a .png image, run:
img2img.sh color_growth_title.psd png Retrieves and reads out loud from the Gibberish Artist Statement Dispenser https://earthbound.io/data/gibberish-artist-statements/index.php (URL subject to change), via the Mac "say" command/utility. MacOS only.
Run without any parameter:
speakArtGibberish.sh Prints all file extensions (types) found in the current directory, and optionally in all subdirectories also.
Run with or without this parameter:
To print only file types from the current directory, pass no parameter to the script:
printAllExtensionsHere.sh To print file types from the current directory and all subdirectories, pass anything as a parameter to the scripr:
printAllExtensionsHere.sh BLARFARFARBARG To store the printout in an array for further use, do a command substitution like this:
allFileTypes=$(printAllExtensionsHere.sh BLARFARFARBARG) Or to pipe the results to a file, do this:
printAllExtensionsHere.sh BLARFARFARBARG > allFileTypesRecursive.txt Runs toTypeFolder.sh for every file of every type in the current directory, and optionally also for all subdirectories.
Run with or without an optional parameter:
Example that will sort all files of every discovered extension from the current directory (for example txt and png) into subdirectories named after those types (for example '/txt' and '/png') :
allToTypeFolders.sh Example that will sort all files of every discovered extension from the current directory and subdirectories (for example txt, hexplt and png) into subdirectories named after those types (for example '/txt', '/hexplt' and '/png') :
allToTypeFolders.sh FLIBFLUB Runs dos2unix on all files found in either the current directory, or optionally the current directory and all subdirectories as well. See USAGE.
The dos2unix utility in your PATH.
To run dos2unix on all files in the current directory, but not on files in subdirectories, run without any parameter:
allDOS2unix.sh To run dos2unix on all files in the current directory and all subdirectories, run with anything (for example the nonense word 'WABYEG') as a parameter:
allDOS2unix.sh WABYEG Updates the index for every file in a repo of .sh and .py types to regard them as executable (in Unix-style file systems that care), and also sets folder and file permissions to avoid permission changes triggering a git change detection.
Run this script without any parameter:
fixRepoPermissions.sh Calls listMatchedFileNames.sh for every file with extension (or file type) $1 in the current directory.
Run with these parameters:
$1 The extension of the file for which you want every file of that type to be passed to listMatchedFileNames.sh.
$2 OPTIONAL. A string declaring one of three modes: 'mode=1', 'mode=2', or 'mode=3'. (Parameter can be with or without surrounding quote marks.) If omitted, defaults to 'mode=1'. Mode 2 causes the script to only list file names and how many matches were found, like this:
fBnhR9Ar.hexplt 8
YWgZmFP3.hexplt 5
Mode 3 causes the script only to print notifications when a file has no match.
To run listMatchedFileNames.sh for every filename in the current directory that ends with .hexplt, run:
listMatchedFileNamesOfType.sh hexplt To run for every hexplt file in mode 2, run:
listMatchedFileNamesOfType.sh hexplt mode=2 For file $1 (parameter to script), prints all other files and directories in the current directory which have the basename of $1 as part of their file name. Optionally recursive, also. For example, if $1 is fBnhR9Ar.hexplt, this script will list all other files and/or folders that contain the string fBnhR9Ar. May be used for example to identify whether render configuration or source files have matched targets derived of their file name (if your process makes render targets have file names that indicate their source). If this script prints anything, there is a match. If it doesn't print anything, there is no match.
Run with these parameters:
To print files and folders in the current directory that include the base name of r8e9E62z.hexplt (which is r8e9E62z) in their file name, run:
listMatchedFilenames.sh r8e9E62z.hexplt To print files and folders in the current directory and subdirectories that include the base name of YWgZmFP3.hexplt (which is YWgZmFP3) in their file name, run:
listMatchedFilenames.sh YWgZmFP3.hexplt foo match, pair, target, orphan, unmatched, unpaired, extension, found
Pandoc wrapper. Converts source document $1 to target document $2 via pandoc. Also copies respective timestamps from source file to target (via cygin/'nix/mac touch command, and binarez_touch on Windows), so that each newly created (converted) document has the same time stamp.
Run with these parameters:
Example that would convert an input file README.md to README.odt:
pandoc2doc.sh source.md README.odt Pandoc doesn't know how to write all formats that can be contrived by simply passing the file extension as a parameter to -t, as this script does.
Makes a directory tree of randomly named folders and possibly subfolders, and random files of certain types (hackable, see NOTES), optionally with terminal unfriendly characters (see USAGE). The result random directory tree is in a subfolder named testFiles. The script deletes that directory tree and re-creates it on each run.
See also FileTool: https://tools.stefankueng.com/FileTool.html
Run with an optional parameter:
Example that will generate a tree with randomly named files and folders, without terminal-unfriendly characters in their names:
makeRNDtestFilesTree.sh Example that will generate a tree with randomly named files and folders, which may also include terminal-unfriendly characters in their names:
makeRNDtestFilesTree.sh POLSDERF Strips many kinds of source code comment markers (leaving everything on the line but the comment marker) from the start of any file, printing the result to stdout. Can optionally overwrite the original file (removing comments from the original file). Note that it removes the comment marker, but not the rest of the comment; so for example any file that has c-style comments like this in it:
// this be a comment yar -- will have that comment marker removed, but the rest of the text remains, like this:
this be a comment yar Run with these parameters:
Example that would take a file named script.sh as parameter one, and remove all the # (number or hash or pound sign) comment markers from it, and print the result to stdout:
commentMarkerStrip.sh script.sh Example that would take a file named script.sh as parameter one, and remove all the # comment markers from it, and overwrite script.sh with the result:
commentMarkerStrip.sh script.sh CHULFOR Finds the oldest commit date for every file in this directory and all subdirectories, and sets the modification date of the file to the date and time of that earliest commit.
Git, and for Windows, ExifTool. For Unix-like environments, GNU coreutils that come with most of them, including touch.
With this script in your PATH, and from the root directory of a repo, run:
setFileTimestampsToEarliestGitCommit.sh If you run this on a large repository (many files), it can take a LOOONNG time. It scans git history for every file in the repo and modifies file change (and for Windows, creation) date stamps for. Every. File. Excluding the .git folder, thank goodness.
This logs time stamps of first commits for each file in a file named firstGitCommitTimeStamps.txt. On subsequent runs, it searches that log first (which hopefully is faster), and if it doesn't find entries, it searches git log, then appends any finds to firstGitCommitTimeStamps.txt.
There's no point to running this script if you don't do so from within a folder of a git repository (the root of the repo or any subfolder).
If you have ridiculously long history with a file, and/or the file was ever renamed, you may want to first run:
git config diff.renameLimit 999999
This script was created for the data archaeology curiosity of listing what files I created first in a git repository, and sorting them by newest first to see what is "newest" in terms of when I first committed it. Also, it tries to track renames in git history.
Adapted from gitDumpAllFileVersions.sh, which is adapted from information it references.
Prints full paths (relative to current directory and subdirectories) of all file types passed as parameters to this script, optionally with sort by newest file modification date first.
Run with these parameters:
Example that will list all files with the extensions .sh, .py and .c:
printFilesTypes.sh sh py c Example that will print many file types, with the custom sort order of most recently modified first per type:
printFilesTypes.sh NEWEST_FIRST sh py pl c cpp bat ahk reg Intended to help check the last frame of an animation render (in a folder of numbered e.g. images), to see if it matches a file outside it. Which only happens as a result of color_growth render scripts I have. Or maybe in other settings. How it does this: finds all folders with the same base file name as all files of type $1 in the current directory, and opens both the check file of type $1 and the last listed file for each matched-name folder. (At this writing, via the start command, which dunnae work on all platforms.)
Run with one parameter ($1), which is the file name of a file expected to be identical to the last file found in a folder with the same base name as that file $1. For example:
checkLastAnimationFrameVSimage.sh 750_from_uGSQNAA6__2020_07_16__05_51_52__6077c6.png Re-encodes input video $1 at quarter resolution.
Run with one parameter, which is the file name of the input video to make a quarter-resolution copy of:
vidXquarter.sh inputVideo.mp4 Result file will be named
Re-encodes input video $1 at half resolution.
Run with one parameter, which is the file name of the input video to make a half-resolution copy of:
vidXhalf.sh inputVideo.mp4 Result file will be named
DEPRECATED. Converts file $1 to Adobe DNG (digital negative), but don't use DNG. At best you have to keep the original raw image around anyway (in case the DNG toolchain becomes outdated), and DNG has not seen wide adoption.
AdobeDNGConverter.exe in your PATH.
Run with one parameter, which is a file name for an image to convert to DNG, e.g.:
img2dng.sh inputFile.cr2 Losslessly copies the audio and video streams of input media file $1 to
ffmpeg.
Run with on parameter, which is the file name of the media file to copy the streams for, e.g.:
ffmpegFileToMPEGtransport.sh inputFile.mp4 Writes metadata information for image or video etc. $1 to
exiftool
Run with on parameter, which is the file name of the file to dump information for, e.g.:
exiftool_dataDump.sh inputFile.jpg Checks for install of homebrew on Mac and installs it if not detected.
Run the script without any parameters:
installMacHomebrew.sh Searches the operating system PATH for a script file name $1 (parameter 1) and prints the full path to it if found. First tries which, which is a lot faster, and if that fails, searches every directory in $PATH. Can help other scripts find the full path to a script they rely on and use, if they capture and utilize the printed output from this.
Suppose you have a script that calls other scripts which you know are in your PATH, but it's inconvenient and/or impractical to hard-code the full path to those other scripts in your script. On top of that, the which and realpath commands fail to find a file which you know exists in your $PATH on MSYS2 (and maybe other emulated Unix-like environments).
Run this script with these parameters:
For example, if the script you want the full path to is color_growth.py, run this script with only that, like this:
getFullPathToScript.sh color_growth.py If this script finds the full path to it, it will print it, e.g.:
/c/_ebDev/scripts/imgAndVideo/color_growth.py You may exploit this print from any other script that wants that full path, by assigning it to a variable. For example, in another script, you might do this:
pathToScript=$(getFullPathToScript.sh color_growth.py)
python $pathToScript $colorGrowthPyParameters This script was created because the which command apparently doesn't actually search every directory in the $PATH on MSYS2, or if it does, something with it is broken on my setup for some files I try to find with it.
Searches for .sh and .py type scripts in all subdirectories and runs a git command against each of them to git add them with the execute bit set. Solves a problem where files created and normally added to git on Windows don't have that, and some 'Nixy environments properly refuse to execute them without that bit. (In fact, even if they are in your PATH, some or all 'nix environments behave as if they don't even exist if you try to locate them with whereis scriptName.sh or whereis scriptName.sh -- it doesn't even throw an error.)
Hard-code the script types listing in the find command at the start of the script per your preferences, then run this script without any parameters:
gitAddAllScriptsExecBit.sh As this only runs git add against all the script files, you'll need to properly git commit and git push or whatever to actually get them into a remote/shared git repository.
Helps either eliminate or group palettes (.hexplt files) in all subfolders of the current folder (but not palettes in the folder itself) which are perceptually similar (technically and logically: not very different below a threshold). Does this with a custom loop in this script, using other scripts also. To understand how all this works, you must examine the DESCRIPTION and USAGE etc. comments of all the scripts which this script runs.
Hack the global value right after the CODE comment per your want. Then run the script:
reduceSimilarPalettesAssistant.sh runs RGBhexColorSortInCAM16-UCS.py with original file overwrite parameter against all .hexplt files in the current directory (no recursion into subdirectories), comparing them by sorting nearest to black-black-black-magenta first (makes a temporary copy of every hexplt file with black as the first color, runs the comparison, then removes the added color).
getFullPathToFile.sh, RGBhexColorSortInCAM16-UCS.py
With more than one .hexplt file in your current directory, and RGBhexColorSortInCAM16-UCS.py in your PATH, run this script:
allRGBhexColorSortInCAM16-UCS.sh See also allRGBhexColorSortInOkLab.sh' and allRGBhexColorSortInCIECAM02.sh'.
Lists pairs of palettes that are visually similar to each other below a threshold, $1. Operates on a file created by allPaletteCompareCIECAM02.sh, which must be run before this, and creates a list of all image pairs in a directory ranked by dissimilarity from each other. This script filters results from that below float parameter $1. See USAGE for notes on similarity rank float.
First, run allPaletteCompareCIECAM02.sh as instructed in its comments. This is necessary to create the file paletteDifferenceRankings.txt, which this script relies on. Then, run with one parameter:
Example run that will list every palette pair where the difference value is below 0.065:
listPaletteDifferencesBelowThreshold.sh 0.065 runs paletteCompareCIECAM02.py repeatedly, for every .hexplt (list of RGB colors in hex format) pair in the current directory, printing the results to paletteDifferenceRankings.txt.
allPalettesCompareCIECAM02.sh Lists pairs of images that are visually similar to each other below a threshold (parameter $1). Operates on a file created by imgsGetSimilar.sh, which must be run before this, and creates a list of all image pairs in a directory ranked by nearness of similarity to each other. This script filters results from that below float parameter $1. See USAGE for notes on similarity rank float.
First, run imgsGetSimilar.sh as instructed in its comments. This is necessary to create the file imageDifferenceRankings.txt, which this script relies on. Then run this with one parameter, which is a float value between 0 and 1. Image pairs that have a comparison value below that float will be written to a new file: imageDifferencesBelowThreshold.txt
Example run that will list every pair where the nearness value is 0.065 or lower (meaning the computation for how different the images are is 0.065; or in other words, they are 1 minus 0.065 or 0.935 (93.5 percent) or more similar) :
listImagesDifferentBelowThreshold.sh 0.065 Prints count of file type $1 in current directory (CASE-SENSITIVE search), and optionally in all subdirectories.
Run with these parameters:
Example that prints the count of all files that end with .png in the current directory:
count.sh png Example that prints the count of all .hexplt files in the current directory and all subdirectories:
count.sh hexplt FLAERGHBLOR Search is CASE-SENSITIVE: if you give it cr2 it will only find lowercase cr2 extensions. Conversely if you give it CR2 it will only find uppercase. If you want case-insenstive search, hack the script to use the -iname switch with find (instead of -name).
Creates a randomly named file stub for new .sh script development, like this:
start command does anything on your system (like hopefully open the stub in a text editor).Run without any parameter:
newScript.sh Via other scripts, obtains N perceptually unique (vs. one another) color palettes. The last script this calls lists palettes that are perceptually similar (technically and logically not very different below a threshold), for proposed deleting.
Hack the global values right after the CODE comment per your want. Then run the script:
fetch_and_render_Ncolormind_palettes.sh To remove palettes for which you manually delete a rendered PNG (because you don't want the palette), see listUnmatchedExtensions.sh or pruneByUnmatchedExtension.sh.
Makes a video that should not exist, of random pitches paired with random color screens, for random durations, concatenated.
getFullPathToFile.sh, randomFloatsInRange.exe, randomString.sh, RNDcolorAndPitchVid.sh, concatVideos.sh, and whatever utilities the scripts called by this script uses (e.g. ffmepg).
Run with these optional parameters, which have defaults if not provided:
Example that will produce 42 videos, of minimum duration 0.68 seconds and maximum duration 1.24 seconds, and then concatenate them into one longer video:
whatIsLifeAndWhatIsArtAndWhyDidIMakeThis.sh 42 0.68 1.24 The result file is named after a time stamp and the convoluted initials of the title of this script, for example: 2020_07_20__17_51_28__899904000__WILAWIAAWDTVE.mp4
This script is for a highly custom and hard-to-describe scenario. It finds all .cgp and .sh format files which contain the transformed RGB color triplet patterns in a .rgbplt file (which was used with a color_growth_hexplt.sh batch run or runs), for the current directory. See USAGE. It is possible I will fix the root scripting / render design problems that lead to the need of this script, and thereby obsolete this script.
Yes, I could just accept multiple renders from the same origin coordinate color in a batch. But no, it's a creative choice.
Stated problem: I ran and interrupted and resumed color_growth_hexplt.sh repeatedly, resulting in more than one cgp preset for every color in the converted .rgbplt file. I want to narrow it down to one render per color in the palette, deleting any more than one render per color in the palette, keeping the more favorable ones.
This script lists all .cgp and .sh files in the current directory which have a color RGB tuple format (e.g. '[212,123,57]', but without single quote marks) transformed from the original .rgbplt format (e.g. '212 123 57' without single quote marks). I can then find the corresponding .png files rendered from those .cgp presets, and delete all but my favorite cgp and corresponding png render (pruneByUnmatchedExtension.sh will help there) files, keeping the one that I like best.
To get help with that, after you've finished all renders via color_growth_hexplt.sh (after it runs the whole script without doing any more renders, assuming you leave all .rendering stub files intact), run this script with one parameter, which is the .rgbplt file to run this analysis for, e.g.:
findDuplicateCGPcolorRenders.sh collectedColors1.rgbplt Results will be stored in findDuplicateCGPcolorRenders_sh__log.txt.
Fetches a randomly served color scheme from http://colormind.io and converts it to a .hexplt file named after the colors in the palette.
curl and sed from a 'Nixy environment (MSYS2 on Windows may be best) to run this bash script, and the colormind API up and running over the internet. Colormind documented at: http://colormind.io/api-access/ and https://github.com/dmi3kno/colormind
get_colormind_RND_palette.sh results in a new file named something indigestible like:
colormind_dot_io__0C4F4F_D8CBAA_FD7A07_FC2515_720B17.hexplt --with one hex color code per line of those respective color codes in the file name itself.
Finds all files of type $1 (parameter 1), sorts them by date (oldest first), then runs cat against each. Meow.
Run with one parameter, which is a file type (or anything else you can use), without any . before the extension (for example just txt) to pass repeatedly to cat, like this:
catByDate.sh txt To pipe the output to a new file, run it like this:
catByDate.sh hexplt > all_palettes.hexplt Makes a stitched anim from output subdirectories of SVG animation frames, from small_and_simple_things.pde (Processing) output with (for example) the following global variables set to these values:
boolean savePNGs = false;
boolean saveSVGs = true;
boolean saveAllFrames = false (or true!);
boolean saveAllFramesInteractOverride = true (or false!); After you obtain so many images in subdirectories via small_and_simple_things.pde, run this script without any parameters:
stitchedAnimsFromSVGsubdirs.sh See outdented "OPTIONAL" comment for a line of code you may uncomment that will leave all copied svgs (rendered frames) after a certain count out of the render (for example to only render less than a minute's worth of frames).
Uses other scripts to create a video of crossfades through images sorted roughly by next most similar. Can be useful for exploring new abstract art ideas by crossfading similar (or very different!) images, or to explore visual ideas, or just for curiosity.
Examine comments in the scripts this recipe uses to get an idea what goes on here. Run the following commands and scripts as listed in the source code, adapting for your situation. Refer to comments in code, but ffmpegCrossfadeIMGsToVideoFromFileList.sh has comments which are more complete.
Using other scripts, creates an animation where every frame is the next most similar image. Animation is 30fps, but with the source "framerate" for each image at 0.83 FPS (each still in the animation shows ~1.2 seconds).
Examine the source code of this script, and the scripts it calls, to get an idea what goes on here. Run this script with:
next_most_similar_image_anim.sh This was a one-use script; it fixes something that happened before I fixed randomNsetChars.pde to do it. Reads seeds (for randomNsetChars.pde) from seeds.txt in a folder above so many animation frame subfolders, and writes those seeds into text files named after the seeds, in the subfolders. To identify seeds that generated random animation images.
Run this script without any parameter:
move_seeds_into_subfolders.sh Uses several other scripts to make animated bent data art from any series of files of a given extension. The resulting _out.gif (or out.mp4) animation visually represents (as animated data bent art) changes made in a file over time. Every (blown up) pixel in an animation represents a datum from the source file (in an animated series of images).
The dependencies of this script sadly may not all be platform-neutral. They are:
IrfanView
all_data_bend_type2PPMglitchArt.sh
imgs2imgsNN.sh
renumberFiles.sh
ffmpegAnim.sh -- and their various dependencies, and a series of automatically backed up files from a file that was edited AND/OR pulled out of git via gitDumpAllFileVersions.sh.
From a path containing so many incremental (or otherwise saved!) *.bak files, run this script with:
Example that will produce an animation from all files in the current directory that end in .bak:
makeDataBentAnim.sh bak hexplt2rgbplt_as_dataBentAnim, the mp4 will be named hexplt2rgbplt_as_dataBentAnim.mp4.The Processing script randomNsetChars.pde writes the seed for each animation frames collection in the same folder as the frames, in a text file which is also named after the random seed. This script collects those from the subfolders/text files and lists them in a file outside (above) the subfolders, named "seeds.txt". There are reasons I want this script (and not randomNsetChars.pde) to do that.
This wipes out the contents of seeds.txt (before repopulating it) if it already exists.
Run without any parameter:
listSeedsFromRandomNsetCharsPDEanim.sh Creates a markdown image gallery of randomly generated color palettes using "gray math" (where the sum of colors by additive color mixing will make a shade of white). At this writing and possibly forever, the palette rending script (which this calls) that calls another script is not platform-neutral (uses irfanview).
genRandomColorsGrayMathGallery.sh Creates a markdown image gallery of randomly generated color palettes. Not platform-neutral (uses IrfanView).
Run without any parameter:
genRandomColorsGallery.sh Runs ffmpegAnim.sh against every subfolder in the current folder, with default hard-coded values if you provide no parameters, or passing on to ffmpegAnim.sh whatever parameters you provide for this script. Renames the resulting animation videos after the respective subfolders. See USAGE and NOTES.
From a directory with so many subdirectories, each with image series suitable for ffmpegAnim.sh (SEE), run this script. Run it without any parameters to use provided defaults, or pass to this script the same parameters that ffmpegAnim.sh will accept, and this script will pass those to ffmpegAnim.sh for every call of that script, for every subfolder. To use without any parameters, run like this:
ffmpegAnimsDirs.sh runs idiff_all_img_pairs.sh, then gm_average_all_img_pairs.sh (both from the _ebDev repository--both those scripts and their dependencies must be in your PATH), then takes all the results of the former and averages them with all the results of the latter (for more interesting/subtle effects that lighten up too dark results), rendering all those results to the /process_hybrids subfolder. See comments in those first two scripts to learn what they do. NOTE: This script takes a random pair from array A and super-composites it against every image in array B. In contrast, a very similar script, diff_avg_supercomposites.sh, repeatedly supercomposites random pairs from array A and B.
All the following in your PATH, maybe via _ebPathMan:
Also, this script expects images to be in subdirectories named:
image_pairs_diffs
image_pairs_averages First, perhaps run one and/or both of these scripts:
idiff_all_img_pairs.sh
gm_average_all_img_pairs.sh You need the subfolders mentioned in DEPENDENCIES. Then, run this script without any parameter:
diff_avg_supercomposites_nested_loop.sh Repeatedly supercomposites random pairs from array A and B. SEE ALSO diff_avg_supercomposites_nested_loop.sh, which takes a random pair from array A and super-composites it against every image in array B.
Run with one parameter, which is how many supercomposites you want to generate by random selection from two image arrays, e.g. to make 150 of them, run:
diff_avg_supercomposites.sh 150 Creates an animation from all images of type $1 in the current directory, by creating numbered hardlinks to them in a temp subdirectory, moving into that directory, creating the animation, moving it out of the directory, then removing the temp directory.
To create an animation from e.g. all png images in the current directory, run this script this way:
anim_from_imgs.sh png For any other image type specify that (the extension of it without the . in the extension) as parameter $1.
Recursively optimizes all pngs in the current directory and subdirectories, via optpng, preserving full fidelity of original pixels. See NOTE for pngquant option.
optiPNGall.sh NOTE To additionally use pngquant but lose color fidelity in the process, uncomment the line in the for loop which begins with pngquant.
Downloads all images from earthview.withgoogle.com. Yes, that was me querying every possible numbered image at your domain from 0 to 10,000 on Dec 27 2018. I paused substantially between each query, though, so as to not hog your server bandwidth.
wget.
From a directory into which you want all such images to be downloaded, run this script:
get_all_earthviews_from_withgoogle_com.sh Lists all files of type $1 in the current directory for which there are NOT file type $2 pairs (files with the same file name base). Can also search for unmatched directory names for $1 or $2 (see USAGE). May help for e.g. manually re-starting renders from source configuration files (by identifying render configuration files which have no corresponding render target file, and/or missing intended directory target names).
Run with these parameters:
listUnmatchedExtensions.sh sourceFileType missingPairFileTypeToList For example, if you are rendering so many images from color_growth.py preset files (which have the .cgp extension), and want to list all .cgp files that have no corresponding .png file of the same name, you would run:
listUnmatchedExtensions.sh cgp png To count how many unmatched extensions there are, pipe it to wc with a flag:
listUnmatchedExtensions.sh cgp png | wc -l listMatchedFileNames.sh and listMatchedFileNamesOfType.sh.orphan, unmatched, unpaired, no pair, extension, not found, pair
Installs desired msys2 packages and does other customization setup.
Run this script without any parameters:
MSYS2setup.sh This overwrites .ini and bash profile files with settings from the same directory this script is kept in. Those settings include:
Variant of printAllIMGfileNames.sh but for video files. Prints a list of all files matching many video file types in the current directory, and optionally all subdirectories. To create an array from the list, see USAGE.
Run without any parameter:
printAllVideoFileNames.sh To use this from another script to create an array from the output, do this:
allVideoFileNamesArray=($(printAllVideoFileNames.sh)) -- you may then iterate through it like this:
for element in ${allVideoFileNamesArray[@]}; do <something with $element>; done By default, the script only prints files in the current directory, but if you pass any parameter to the script (for example the word 'BROGNALF'), it will also (find and) print image file names from subdirectories:
printAllVideoFileNames.sh BROGNALF Because some tools are silly and create files with uppercase extensions, this script searches for both lowercase and uppercase extensions of every file type in its list.
Variant of printAllIMGfileNames.sh and printAllVideoFileNames.sh but for sound files. (Doesn't even attempt exhaustive list of sound file types, at this writing.) Prints a list of all files matching many sound file types in the current directory, and optionally all subdirectories. To create an array from the list, see USAGE.
Run without any parameter:
printAllSoundFileNames.sh To use this from another script to create an array from the output, do this:
allSoundFileNamesArray=($(printAllSoundFileNames.sh)) -- you may then iterate through it like this:
for element in ${allSoundFileNamesArray[@]}; do <something with $element>; done By default, the script only prints files in the current directory, but if you pass any parameter to the script (for example the word 'BROGNALF'), it will also (find and) print image file names from subdirectories:
printAllSoundFileNames.sh BROGNALF Because some tools are silly and create files with uppercase extensions, this script searches for both lowercase and uppercase extensions of every file type in its list.
Prints a list of all files matching many image etc. file types in the current directory, and optionally all subdirectories. To create an array from the list, see USAGE.
Run with these parameters:
Example command to find and print files from the current directory only:
printAllIMGfileNames.sh Example command to find and print files from the current directory and all subdirectories:
printAllIMGfileNames.sh BROGNALF To find and print file names from subdirectories and print full paths:
printAllIMGfileNames.sh BROGNALF 'RETURN OF BROGNALF' To find and print file names only in the current directory, and print full paths:
printAllIMGfileNames.sh NULL 'RETURN OF BROGNALF' USE FROM ANOTHER SCRIPT. To use this from another script to create an array from the output, do this:
allIMGfileNamesArray=( $(printAllIMGfileNames.sh) ) -- you may then iterate through it like this:
for element in ${allIMGfileNamesArray[@]}; do <something with $element>; done You can also use the positional switches for this script to create an array with particular properties you want; for example to create an array that lists all supported image files in all subdirectories, with full paths, you would run:
allIMGfileNamesArray=( $(printAllIMGfileNames.sh BROGNALF 'RETURN OF BROGNALF') ) For all files of type $1 in the current directory, searches for files of type $2 that have the same base file name (file minus extension) which reside up to $3 directories up from the current directory, and moves them into the current directory. Checks for and will not move files to the current directory that would clobber (overwrite) files, and logs those duplicates to a file it notifies of at end of script run.
Run with these parameters:
Suppose you've got some files named rnd_43aB.png and rnd43Cd.png in the current path, which you rendered from source files rnd_43aB.flame and rnd43Cd.flame, which are two directories up from the current directory, and you want to move those matching .flame files into the current directory. To move those matching .flame files from up to two parent directories down to the current directory, run this script with these parameters:
getFilePairs4Up.sh png flame 2 This will result in all those matching .flame file names being moved from up to two directories above into this directory.
The script will not overwrite existing files in the current directory with identically named files from any parent directory. If it finds duplicate file names, it will log the full paths to the duplicates to a file named something like this: getFilePairs4Up_run_EB213A_log.txt
To override that safe behavior and force overwrite existing files, pass anything for parameter $4. Adapting the example to that purpose, the command to do that would be:
getFilePairs4Up.sh png flame 2 KRIDTALB
Converts all of many video media container (file) types in the current directory to mp4 containers, losslessly; there is no recompression: it directly copies the video streams into a new container. It also copies the file timestamps (including Windows-unique ones) and relevant metadata from the original file to the converted target file, via another script. For options for lossless video but lossy sound, see NOTES.
ffmpeg, GNU touch, copyMetadataFromSourceFileToTarget.sh
Run without any parameters:
allVideo2mp4Lossless.sh If you have a .mov container as source with PCM sound, you may get an error copying (maybe mp4 files can't have PCM audio??). In that case try copyVidLosslessEncodeSound.sh or copyVidLosslessEncodeSoundAllType.sh
Helps discard files in the current folder (non-recursive) with a given extension that have no companion file with the same file but a different extension. Does this by moving the identified files into a temp folder for review before permanent and manual deletion. See NOTES.
If you use this on files with unintended dissimilar base file names such as thisFractalRenderFlame.flame.png (two periods in the file extension), you will lose work!
See the above WARNING!
Scenario: suppose you have a set of render source or configuration files (in my case it might be files of extension .cgp, or .hexplt) for which you have rendered targets (e.g. .png images of color growth script settings or palettes). You may then go through and delete the .png results you don't like, and run this script to move all the associated .cgp or .hexplt files into a temp folder. You can then review the files in the temp folder and manually delete them. The parameter format in more of a sentence form is:
pruneByUnmatchedExtension.sh FileTypeToMove ifNoMatchedFileOfThisType For example, this command will sort every file with the extension .hexplt that has no same-named file (same basename) with a .png extension into a temporary folder for manual analysis and deletion or recovery:
pruneByUnmatchedExtension.sh hexplt png Here is a more detailed example. Suppose you have so many .ppm files which you have converted to .png:
img_01.ppm
img_01.png
img_02.ppm
img_02.png
img_03.ppm
img_03.png
img_04.ppm
img_04.png Suppose you want to delete some of the source .ppm files because you viewed the pngs resulting from converting them, and you don't like them. First delete the rendered pngs you don't like, so that you have these files left:
img_01.ppm
img_02.ppm
img_02.png
img_03.ppm
img_03.png
img_04.ppm Then run this script with parameters that tell this script $1 the source file extension to search for matching file names with extension $2, where $1 will be deleted if no file with extension $2 is found.
Example command:
pruneByUnmatchedExtension.sh ppm png When this script is run, these ppm files that had no png with the same base file name will be sorted into a folder with a name that includes some random characters, like ___delete_candidates_tmp__BB01BF75857DDFE0:
img_02.ppm
img_02.png
img_03.ppm
img_03.png You may then manually check them and delete them after you are sure you don't want them.
orphan, unmatched, unpaired, no pair, extension, prune, delete, not found, pair
Prints the full path to a .hexplt file if found, else prints nothing.
A file ~/palettesRootDir.txt (and in that location--the root of the home dir), which has the path to a repository of palettes (see the _ebPalettes repository and its script, createPalettesRootDirTXT.sh).
With this script in your PATH, run it with one parameter, which is the file name of a palette (such as a .hexplt file name, which is a list of RGB color codes in hex format, one color per line) you wish to locate. For example:
findPalette.sh turtleGreenTetradicEtcHex.hexplt You can utilize the printout from this script in other scripts by using command substitution to assign the result to a variable, this way:
fullPathToHexplt=$(findPalette.sh turtleGreenTetradicEtcHex.hexplt)
-- which results in the full path to turtleGreenTetradicEtcHex.hexplt being stored in the variable $fullPathToHexplt.
turtleGreenTetradicEtcHex.hexplt but pass turtleGreenTetradicEtcHex or turtleGreen, it will fail.~/palettesRootDir.txt, it doesn't let you know. You have to figure that out for yourself, but a clue is that this script prints nothing.Loads an unencrypted password from your Unix profile root directory (which you should not do unless you're confident no one can access it, and probably not even then), and uses that to authenticate to a remote terminal via putty.
Run with one parameter:
For example:
puttyAuth.sh beeblebrox@panic.org Remote terminal, authentication, remote shell, putty, ssh
Moves all files of type $1 (png, jpg etc. -- configurable by first parameter) from all directories one level down up to the current directory. Optionally copies from all subdirectories (recursive). See USAGE. Will not move files from lower directories already found in the current directory. SEE ALSO copyTypeUp.sh (identical except it copies files).
Run with these parameters:
Example that will move all files of type .hexplt from directories one level down (subdirectories, but not their sub-directories) to the current directory:
moveTypeUp.sh hexplt Example that will move all files of type .png from all subdirectories (recursive) do this directory:
moveTypeUp.sh hexplt FALSNARF Copies all files of type $1 (png, jpg etc. -- configurable by first parameter) from all directories one level down up to the current directory. Optionally copies from all subdirectories (recursive). See USAGE. Will not copy files from lower directories already found in the current directory. SEE ALSO moveTypeUp.sh (identical except it moves files).
Run with these parameters:
Example that will copy all files of type .hexplt from directories one level down (subdirectories, but not their sub-directories) to the current directory:
copyTypeUp.sh hexplt Example that will copy all files of type .png from all subdirectories (recursive) do this directory:
copyTypeUp.sh hexplt FALSNARF For the current directory, takes all files of type $1 (parameter 1), and produces a subdirectory of incrementally numbered (file name) hardlinks to them, optionally randomly shuffling the file list before generating hardlinks. Does not directly do this; creates a script which you may execute to do this.
Run with these parameters:
Example that will generate a script which can create numbered hardlinks (in a subdirectory) to all png files in the current directory:
genRandomHardlinksSubdir.sh png Example that will generate a script which can create numbered hardlinks (in a subdirectory) to all png files in the current directory, with the source files list shuffled before numbered hardlink creation (to randomize thier order in the numbered hardlinks) :
genRandomHardlinksSubdir.sh png SNARFBLURN To see what the subfolder name will be, examine the generated script as this script prompts you to after it runs.
For all files of a given type (parameter $1) in the current directory (and optionally all subdirectories), moves them into a new subdirectory (in the immediate folder) named after that type. Creates that subfolder only if it does not already exist, and moves files to them only if they do not already exist. SEE ALSO allToTypeFolders.sh.
Run with these parameters:
Example that will sort all files with the extension .png into a new subfolder named png/ :
toTypeFolder.sh png Example that will sort all file with the extension .hexplt in the current folder and all subfolders into a new directory named /hexplt:
toTypeFolder.sh hexplt EKTHELPOI Rips frames from a video file to 7-digit padded number pngs (animation frames).
Run with one parameter, which is the file name of the input video to rip frames from, for example:
ffmpegRipAllFrames.sh inputVideo.mp4 Result frames are in a folder named
Prints unique presets from an XML file containing multiple Filter Forge filter user presets (which XML files are stored in a "My Presets" user folder with a Filter Forge install). This printout may be pasted over the entire original preset list again and saved. Has the effect of deleting duplicate presets (which appear when I merge presets by hand from multiple sources).
xmlstarlet installed in your PATH as 'xml', sed.
Suppose you have two or more different versions of a Filter Forge user preset file (e.g. Library_14934.xml) from different machines (from the "My Presets" subfolder in the Filter Forge user data folder). If you cut and paste all of the
Steps to accomplish this:
In Filter Forge, navigate to the filter you wish to merge presets from other copies of Filter Forge into. Right-or-alt-click it and click "Locate File." This will open the folder where the .ffxml file resides, with the file highlighted. This is our "preset merge file," and for our examples it is named Library_14934.xml.
Back up this preset merge file somewhere completely other than the Filter Forge user or install folder, to somewhere safe you can restor the file from if things go wrong.
Open the preset merge file and your other copies of the same file (otherwise known as user preset files) from other machines or installs of Filter Forge.
Cut and paste the
Restart Filter Forge, open the preview for the filter (the preset merge file Library_14934.xml), and open and test render different presets to be sure everything is functioning OK. You may notice here that there are identical presets. That's the problem we're solving.
Close Filter Forge.
To deduplicate the collated filters from the preset merge file, run this script with the preset merge file as the only parameter, and pipe the printed result to a new file, like this:
sort_uniq_FFXML_user_filters.sh Library_14934.xml > deduped_elements.xml
Cut and paste the deduplicated presets from the new file deduped_elements.xml over the elements in the preset merge file Library_14934.xml. Replace all of the
At this writing, because Filter Forge doesn't detect stale preset image caches that resulted from operations outside of Filter Forge. It goes right on using the same cached images even if presets were reordered/deleted/inserted outside the program. This means you'll have wrong preset previews. As a workaround, delete the thumbnail data file named after the filter you deduplicate. You'll find it in the /System/Thumbnails subfolder of the user folder for your Filter Forge install.
Start Filter Forge and verify that it has all the new presets but no duplicates.
Once you know that worked, discard the new file from which you merged so many deduplicate presets into the preset merge file.
repeatedly runs color_growth.py with a new --RANDOM_SEED and one custom --CUSTOM_COORDS_AND_COLORS coordinate/color each run; the colors are from a .hexplt file passed to the script; other things per preferences customizable via parameters. Also, it creates stub .rendering files when it starts a render, to enable concurrent runs (of more than one instance of this script running at the same time, but without duplicating work). See USAGE.
Run with at least three required parameters, and optional additional parameters:
--WIDTH (as expected by color_growth.py) of intended render(s).--HEIGHT (as expected by color_growth.py) of intended render(s).color_growth.py --CUSTOM_COORDS_AND_COLORS switch), surrounded by single quote marks, for example '1799,1799'. (Note that while the --CUSTOM_COORDS_AND_COLORS switch expects parenthesis around those values (among other things), it may break it to do that here. Don't use parenthesis here). If not provided, the x,y values are highly randomized (read on). If you need to use $5 but don't want to "use" $4 (leave it at default), pass the keyword NULL as $4.color_growth.py, surrounded by double quote marks, for example '--WIDTH 3600 --HEIGHT 3600 --RSHIFT 8 --BG_COLOR [255,0,255] --BORDER_BLEND True --TILEABLE True --SAVE_EVERY_N 0'. If $5 is not provided, no additional parameters are used, or in other words the variable that controls them is not even set; it is effectively empty as far as the script is concerned). If $5 is provided, you may be able to use --WIDTH and --HEIGHT parameters in it if they wouldn't cause errors vs. $2 and $3. (I suppose only if in $5 they are larger than $2 and $3?) If you want to use $6 but not $5, pass the keyword NULL as $5. NOTE that any use of the --GROWTH_CLIP flag must surround the tuple (the parenthesis) with a single quote mark (while the entirety of $5 itself must be surrounded by double quotes). For example: "--RSHIFT 3 --GROWTH_CLIP '(2,6)' --SAVE_PRESET False"For every color in the .hexplt file ($1), the script will make a color_growth.py render.
EXAMPLE RUN with only a palette, and width 1920 x 1080:
color_growth_hexplt.sh RAHfavoriteColorsHex.hexplt 1920 1080 See comments in hexplt2rgbplt.sh for expected PATH for any .hexplt file ($1).
If $4 is passed as RND_BASIC_POS, for every color_growth.py call (render), the script will randomly choose from any of these six origin coordinates: a hard-coded upper-left, upper-right, lower-left, lower-right, center, or a random coordinate (it will do the math to determine any of them). If $4 is passed as a string matching the Python tuple format (but surrounded by single quote marks), e.g. '(100,50)' (where the first number is an X (across) coordinate, and the second number is a Y (down) coordinate), the script will use that as the coordinate component of the --CUSTOM_COORDS_AND_COLORS on every run it does of color_growth.py. If $4 is not provided, or is passed as the keyword NULL, the script will choose a random coordinate from the whole image range (anywhere within WIDTH and HEIGHT) each render.
EXAMPLE RUN with the same parameters but also using the RND_BASIC_POS keyword for $4:
color_growth_hexplt.sh collectedColors1.hexplt 1920 1080 RND_BASIC_POS EXAMPLE RUN that would cause the script to use a specific origin coordinate for every render (not RND_BASIC_POS):
color_growth_hexplt.sh collectedColors1.hexplt 1920 1080 '(2027,400)' EXAMPLE RUN that would cause the script to choose a random coordinate from the whole canvas range each render:
color_growth_hexplt.sh collectedColors1.hexplt 1920 1080 NULL EXAMPLE RUN that passes custom parameters (with different values than previous examples for the other parameters) -- NOTE that for parameter $5, the tuple passed with --GROWTH_CLIP must be surrounded by single quote marks, but the entire parameter $5 must be in double quote marks to distinguish it syntactically vs. the single quote marks:
color_growth_hexplt.sh collectedColors1.hexplt 9600 2400 '(4800,1280)' "--RSHIFT 3 --GROWTH_CLIP '(2,6)' --SAVE_PRESET False" color_growth.py, this script creates a .rendering file named after the RGB decimal values for the given render, e.g. RGB_COLORS__168-230-207.rendering. When you're done with a batch, you may delete all the .rendering files (they are intended to be temporary). The intent of these .rendering files is for concurrent runs of this script to check for them, and not duplicate work on a render if it finds one. How that works is that the script checks for a .rendering stub before it would otherwise make a render with that color, and skips the render if it finds an existing (match) .rendering file. You can do multiple simultaneous batch renders (exploiting multiple processors/threads) this way too. To interrupt and resume a batch, keep the .rendering files. If a render was interrupted, you may resume it by deleting the associated .rendering file, then run this script.color_growth.py internally zero-indexes coordinates (1 is 0), pass WIDTH and HEIGHT ($2 and $3) as the actual human-indexed (counting starts from 1, or natural numbers) values, because color_growth.py does the zero-indexing adjustment internally.runs color_growth.py once for every .cgp preset in the current directory and all subdirectories. creates .rendering temp files of the same name as a render target file name so that you can interrupt / resume or run multiple simultaneous renders.
Run with or without these optional parameters:
An example that uses parameter $1:
color_growth_cgps.sh '--WIDTH 850 --HEIGHT 180 --SAVE_PRESET False --SAVE_EVERY_N 7150 --RAMP_UP_SAVE_EVERY_N True' An example that uses parameters $1 and $2:
color_growth_cgps.sh '--WIDTH 850 --HEIGHT 180 --SAVE_PRESET False --SAVE_EVERY_N 7150 --RAMP_UP_SAVE_EVERY_N True' PLIBPLUP An example that uses parameter $2 but leaves it to use the defaults for $1:
color_growth_cgps.sh '' PLIBPLUP This is designed to run in multiple simultaneous batch jobs, for example from multiple computers reading and writing to a network drive, or from one computer with many CPU cores, which will allow multiple simultaneous runs of renders if it does not load the CPUs too much. To accomode multiple simultaneous runs, the script does this:
Produces varieties of a color growth. Calls color_growth.py N times (per $1) for preset $2, randomly varying the seed with each run, resulting in so many renders that all have the same setting but a different seed.
From a directory with a .cgp preset for color_growth.py, run with these parameters:
--RANDOM_SEED value each time.Example that would produce 10 renders of the given preset:
color_growth_1cgp_many_variants.sh 10 colorGrowth-Py-scarlet-orange.cgp Runs color_growth_hexplt_multiColor.sh (note that is ~hexplt, where this is ~hexplts) repeatedly, for every .hexplt file in the current directory, with customizable settings for how many colors to use, and width, and height (defaults used if no settings provided). See USAGE.
See the DEPENDENCIES section of color_growth_hexplt_multiColor.sh.
With so many .hexplt palette files of your liking in the directory you want this to work on, run this script, with these optional parameters, which all default to hard-coded values if you don't pass them (examine the parameter parsing right after the CODE comment) :
Example command that will use 15 colors from each palette, and also make 5 renders for each, of width 1920 pixels and height 1080, passing the extra switches '--RSHIFT 1 --SAVE_EVERY_N 0' :
color_growth_hexplts_multiColor_Ntimes.sh 15 5 1920 1080 '--RSHIFT 1 --SAVE_EVERY_N 0' Takes a .hexplt source file $1 and a number of colors to choose from it $2, and runs color_growth.py with the --CUSTOM_COORDS_AND_COLORS option with those colors (converted to RGB vals via hexplt2rgbplt.sh), optionally with other CLI options. See USAGE.
color_growth.py, Python 3, hexplt2rgbplt.sh
Script expects one parameter minimum, and two or three optionally:
EXAMPLE COMMAND, with number of colors in palette used:
color_growth_hexplt_multiColor.sh palette.hexplt ALL ANOTHER EXAMPLE COMMAND, using 5 colors of the palette, and adding custom parameters :
color_growth_hexplt_multiColor.sh palette.hexplt 5 '--RANDOM_SEED 817141 --TILEABLE True --SAVE_EVERY_N 140' ANOTHER EXAMPLE COMMAND, randomly using from 13 to 100 colors (and if the palette has only 5 colors, the script will randomly select colors from the palette until it has picked however many colors it decides to pick between 13 and 100) :
color_growth_hexplt_multiColor.sh palette.hexplt 13-100 '--RANDOM_SEED 817141 --TILEABLE True --SAVE_EVERY_N 140' This script sets the first randomly selected color drawn as --BG_COLOR.
To pass for example the following parameter, do not enclose the parenthesis with double or single-quotes, as you must ordinarily on Windows and maybe other platforms:
--GROWTH_CLIP (1,4)
If you don't specify a GROWTH_CLIP, this will generate a random one in the range of what color_growth.py will accept.
Calls get_rnd_CCC_for_color_growth.py to get values to pass to color_growth.py's --CUSTOM_COORDS_AND_COLORS for a given image, then does that. The effect is to create a .cgp preset for and run a render to get color_growth.py's effect for so many coordinates of an arbitrary image.
Run with these parameters:
Example command:
--RSHIFT 2 --SAVE_EVERY_N 1440 --RAMP_UP_SAVE_EVERY_N True If $3 is omitted default hard-coded values may be used (see "$3" parameter handling section of code).
Example to obtain 2,355 random coordinates and colors from those coordinates from a .png image, with the extra parameters for $3:
call_get_rnd_CCC_for_color_growth-py.sh deep-indigo-preset_lost.png 2355 '--RSHIFT 2 --SAVE_EVERY_N 1440 --RAMP_UP_SAVE_EVERY_N True' You may hack the additionalParams variable at the start of the script for whatever default purposes you may want, but you might be better off calling this script with another script that customizes that, as it would override the defaults in this script if you change them in this script.
Takes a flat file list of display (and/or multi-display) resolutions in format NNNNxNNNN, one per line, and calculates and prints their aspects and number of pixels to:
export_resolutions_list_with_aspects.csv Sorted by:
aspect, width, height, number of pixels. Obtain such a list here:
https://graphicdesign.stackexchange.com/a/137727/46736
In my files I have that in export_resolutions_list.txt
Such a list in the current directory, named export_resolutions_list.txt
Run this script without any parameters:
resolutionsList2details.sh This script sorts and deduplicates the input list, overwriting it in-place, sorting by numeric rank largest first.
Iterates over all directories in a hard-coded array and runs git pull for each. Script enforces only calling it with a local script call (./pa.sh), and not from PATH search and find of this script.
Copy this script (pull_all.sh) to a folder containing some git repository local clones, and rename it to pa.sh. Then in the copied script, alter the repo_directories array (in the code) for your wishes. Then, to pull all those clones automatically, open your terminal to that directory (or cd to it), and run this script:
./pa.sh ./pa.sh (as a script file in your immediate terminal directory). If this script is in your PATH and you try to run it with pa.sh, it will detect that, and notify you of the problem and exit.Counts all files of type $1 in the current directory, then deletes every one that is not a multiple of $2 minus one.
With this script in your PATH, run it with these parameters:
Example which will delete every png image in the current directory that is not a multiple of 3:
decimateFileType.sh png 3 This script should not exist. I hope I never need to use it again.
Scenario: you have a lot of preset files of type $1 (say, .cgp) from which you have rendered target files $2 (say, .png). However, because you are silly and didn't ensure the rendered files have names that give you any clue what preset or source they were rendered from, you have no way of knowing which presets you want to dispose of also. This script attempts to solve that problem through guesswork.
It assumes:
ONLY RUN THIS BATCH ON COPIES of the files, then verify the renames are correct. Why? because it renames all $2 to match all $1 on that assumption, which could be wrong.
Run with these parameters:
An example command that where the source files are .cgp format and the targets are .png:
renamePairedFileTypeToMatch.sh cpg png To express that in variables that read more like a sentence, but this would not be a practical command, because you would never have extensions with those names:
renamePairedFileTypeToMatch.sh sourceExtension targetExtensionMysteryFiles Installs all the Python modules I commonly use.
Run without any parameter:
installUsedPythonModules.sh Uses ImageMagick montage to pack all images of type $1 in the current directory into a montage combining all their resoluition and then some, with options for numbers of tiles across and approximate result pixels across (those options are automated if omitted). Tiles are padded with a dark gray, and the entire result image is padded with a darker gray.
Imagemagick / montage + convert
Run the script with these parameters:
Result file will be named __montage_<name_of_current_directory>.
Example that will create a montage from all png files in the current directory, with number of tiles across auto-decided, and each tile ~800 px wide:
autoMontageIM.sh png AUTO 800 An example command that will accomplish the same but set the tiles across to 9:
autoMontageIM.sh png 9 800 An example that produces the same and with the montage at the full size of all images combined plus padding:
autoMontageIM.sh png 9 800 FULL Converts a .hexplt format palette (a list of RGB colors in hex format) to a list of RGB values. See also rgbplt2hexplt.sh.
Run this script with one parameter, which is the .hexplt format file to convert, e.g.:
hexplt2RGBplt.sh RAHfavoriteColorsHex.hexplt If you have a file ~/palettesRootDir.txt with a root path to search for .hexplt files in it, this script will search all paths below that root folder for the file, IF the file is not in the same directory you run this script from. If the file is in the same directory, it uses it from the same directory.
Resizes an image of type $1, in the current directory, by nearest-neighbor method, to target format $2, with the longest edge scaled up (or down!) to pixels $3. The shortest edge is scaled to maintain aspect, but that can be overriden to change aspect, with $4. Nearest neighbor method will keep hard edges, or look "pixelated." Uses GraphicsMagick, unless the file is ppm or pbm format, in which case it uses IrfanView (which to my knowledge is Windows only).
GraphicsMagick and/or IrfanView, both in your $PATH.
Run with the following parameters:
Example command that will scale the longest edge of a pbm to 640 px (by nearest neighbor method), and scale the other edge automatically to whatever length will maintain the original aspect, and output to a png image:
img2imgNN.sh input.pbm png 640 OR, to force a given longest and shortest dimension for a ppm:
img2imgNN.sh input.ppm png 640 480 Converts all FontForge-compatible files in the current directory of type $1 to type $2.
Run with these parameters:
Example that converts all .sfd files to .otf:
FontForgeFonts2Fonts.sh sfd otf Prints all subdirectories (and their subdirectories) in the current directory, relative to the current directory (so sub-subdirectories etc. will show their parent, then /).
Call this script with these parameters:
find is) will be listed.Example command to print folders of unlimited depth:
listAllSubdirs.sh Example command to search and print folder names only the current directory (one level deep) for folders:
listAllSubdirs.sh FLORFELNAUT To call this from another script and create an array of the output, do:
allSubdirectoriesArray=( $(listAllSubdirs.sh) )
Taken from (original at) : from https://github.com/ryuichiueda/TomoTool
The MIT License. See copy of it beneath the CODE comment in source code.
See also:
https://dev.twitter.com/oauth/overview/authentication-by-api-family
https://dev.twitter.com/oauth/overview
https://dev.twitter.com/oauth/overview/creating-signatures
If you want to see detailed logs, set an environment variable "DEBUG" to 1 or something.
Correct. I don't know yet.
Optimizes all svg files in the current directory, and writes results to
Python with scour installed and in your PATH via this command:
pip install scour From a path with svg files you wish to optimize and clean up, run this script:
svg_scour_all.sh NOTE THAT get_color_gradient_OKLAB.js is recommended over this script for color gradients. This script repeatedly runs getNshadesOfColorCIECAM02.py for every color in a .hexplt file $1, with optional $2 colors (default hard-coded number may be overridden with optional 2nd parameter).
Bash, and all the dependencies of the python script this calls.
With this and the python script it runs, and the .hexplt file you'll work against all in your PATH, run this script with these parameters:
Example that will produce 7 shades from every color in inputPalette.hexplt:
getNshadesOfColorsCIECAM02.sh inputPalette.hexplt 7 get_color_gradient_OKLAB.js is recommended over this script for color gradients.-b 100, referring to the Python script.Repeatedly runs getNchromasOfColorCIECAM02.py for every color in a .hexplt file $1, with optional $2 colors (default hard-coded number may be overridden with optional 2nd parameter).
Bash, and all the dependencies of the python script this calls.
With this and the python script it runs, and the .hexplt file you'll work against all in your PATH, run this script with these parameters:
Example that will produce 7 shades from every color in inputPalette.hexplt:
getNchromasOfColorsCIECAM02.sh inputPalette.hexplt 7 To hack other parameters modify the command=".." assignment in the script directly, for example to -b 100, referring to the Python script.
Via ImageMagick, generates image renders of the text contents of all .txt files in the path from which this script is run, with the images named after the source text file names. Dimensions, font point size, and font are customizable via parameters; if not provided, defaults are used. Will not clobber render targets that already exist.
ImageMagickgetFullPathToFile.shWith this script in your PATH, run it with these parameters:
To use defaults, run the script without any parameter:
texts2imgs.sh Example that will produce a 1920x1080 image with font point size 120, using NotoSerif-Regular.ttf:
texts2imgs.sh 1920x1080 120 NotoSerif-Regular.ttf Faster but more patterned variant of randomNSetChars.sh. Prints N ($1) groups of characters randomly chosen from hackable string CHARSET. If parameter 1 not provided, a default number is used. Optionally dynamically makes CHARSET a random subset of itself. Core function taken from: https://stackoverflow.com/a/26326956/1397555
A 'Nixy environment with seq, shuf, and printf, printf and your file system able to handle the block characters or whatever else you might hack into CHARSET.
On MacOS it may not properly display and wrap printed characters.
Run with one these parameters:
Examples:
To generate and print 12 groups of characters randomly chosen from CHARSET, run:
randomNsetCharsAlt.sh 12 To generate and print 100 groups of characters using a random subset of CHARSET, run:
randomNsetCharsAlt.sh 100 CHORFL To generate and print 15000 groups of characters using a random subset of CHARSET, and wait 0.2 seconds between every print of characters, run:
randomNsetCharsAlt.sh 100 CHORFL 0.7 randomNsetChars.sh for some ideas.See the OPTIONAL comment to possibly cause the script to sleep for a set amount of time between prints of character groups.
Returns approximately N ($1) characters randomly chosen from hackable string CHARSET. If parameter 1 not provided, a default number is used. DEPRECATED; use randomNsetChars.py instead: it is far faster, can write to files, and produces unique subsets.
A 'Nixy environment with seq, shuf, and printf, printf and your file system able to handle the block characters or whatever else you might hack into CHARSET.
Run this script with one parameter, which is the number of characters desired, and pipe the output to a text file, like this:
randomBlockCharsString.sh 800 > block_chars_art.txt The script by default prints hard newlines after 72 characters per line. To override that e.g. with 60 characters, provide that as a second parameter:
randomBlockCharsString.sh 800 60 > block_chars_art.txt Permanently deletes all files in path $1 (parameter one, intended to be a different directory from the one you run this script in) which have the same file name as any file in the directory this script is run from. Warns you and asks you to type password to confirm.
Pass one parameter, which is another path, e.g.:
deleteDuplicateFileNamesAtOtherFolder.sh /c/whydothosefilesexist/there Takes the second page of a screenplay (ostensibly one that has a title page) and generates
a .png format image preview of it, named <fileNameNoExt>_preview.png
gm)Run with one parameter, which is the file name of the screenplay to get an image excerpt of:
pdfScreenplayPreview.sh screenplay.pdf Converts a fountain format plain text file screenplay into a fountain file with semantic linefeeds (AKA ventilated prose or sense lines) joined. Result is printed to <inputFileBasename>_unventilated.fountain. Adapted from:
fountain2pdf.sh Run with one parameter, which is the input fountain file:
fountain2fountain.sh input-file.fountain This script expects everything after the title page of the screenplay (the actual screenplay body text) to start with a line that starts and ends with "> FADE IN:" (but without the quote marks), and you may expect this script to not work if that is not the case.
This script sets a variable named targetFountainFileName, which is a string variable for the target file name. If you call this script from another script, via source, like this:
source fountain2fountain.sh input-file.fountain
-- then the set variable will be available to the calling script after this script terminates, as source does magicky things to cause that. You can then use that variable in the calling script to do other things with that file name.
If you have text connecting punctuation like a double-dash (for emdash) :
one phrase--another phrase
--and you break the connecting punctuation over lines of ventilated prose, like this:
one phrase--
another phrase --the script will add a space in the punctuation when it joins lines, which is probably not what you want. To avoid this, don't break connecting punctuation over lines: keep it on the same line.
Runs toOldestWindowsDateTime.sh for all files in the current directory, and optionally all subdirectories.
To run toOldestWindowsDateTime.sh for all files in the current directory (and not subdirectories), run this script without any parameter:
allToOldestWindowsDateTime.sh To run toOldestWindowsDateTime.sh for all files in the current directory and all subdirectories, run with any parameter, for example the word 'WABYEG':
allToOldestWindowsDateTime.sh WABYEG Uses cputhrottle (Mac) to throttle process $1 (1st parameter) to use max CPU percent $2 (parameter 2)
cputhrottle: https://medium.com/@sbr464/limit-dropbox-and-others-on-macos-from-taking-100-cpu-877266df104d -- download and extract it from the zip file, then run these commands to install it:
sudo mv ./cputhrottle /bin
sudo chmod +x /bin/cputhrottle Run with these parameters:
Example
cputhrottleProcessPercent.sh Python 40 You may need to precede the run of this script with sudo to get super admin permissions for it to control a process, like this:
sudo cputhrottleProcessPercent.sh Python 40 It seems that it doesn't "take" the first time you run this script preceded by sudo if it stops and asks you for a password. Run this script preceded by sudo again, and it will take.
runs RGBhexColorSortInCIECAM02.py with original file overwrite parameter against all .hexplt files in the current directory (no recursion into subdirectories), comparing them by sorting nearest to black-black-black-magenta first (makes a temporary copy of every hexplt file with black as the first color, runs the comparison, then removes the added color).
getFullPathToFile.sh, RGBhexColorSortInCIECAM02.py
With more than one .hexplt file in your current directory, and RGBhexColorSortInCIECAM02.py in your PATH, run this script with these parameters:
RGBhexColorSortInCIECAM02.py to sort perceptually by next nearest color to this color). If provided, it must not have the '#' symbol at the start of the sRGB color (for example, to use a magenta color, just pass f800fc, not #f800fc). If not provided, no arbitrary first color parameter will be passed to RGBhexColorSortInCIECAM02.py.Example that will sort the colors in all .hexplt files in this directory by next nearest color starting with the first color in each .hexplt file:
allRGBhexColorSortInCIECAM02.sh Example that will sort the colors in all .hexplt files in this directory by next nearest color starting with the arbitary sRGB color #0a000a, a magenta black:
allRGBhexColorSortInCIECAM02.sh 0a000a See also allRGBhexColorSortInOkLab.sh and allRGBhexColorSortInCAM16-UCS.sh.
Cleans up local git branches which have no remote. Also cleans up local expired or orphaned git objects. Assumes the main trunk branch is named master. See comments in source code for reference URLs.
Run without any parameters:
git-cleanup.sh blink1 USB LED programming script to create work/break timer: slower speed random light changes for work period, faster speed random color changes for break period.
Stripped down version of blink1BreakTimer.sh which instead writes instructions to a blink device (flashes the device memory) so that it will play the pattern when connected to USB power (when it is not connected to a computer).
With blink1-tool in your PATH (https://blink1.thingm.com/blink1-tool/), run this script:
blink1BreakTimerWriteToBlink.sh SAVES a boot pattern to a blink1 (USB-powered LED) device via blink1-tool (Mac).
reference:
https://github.com/todbot/blink1-tool/blob/master/scripts/blink1-pattern-fill.sh
https://github.com/todbot/blink1-tool/blob/master/scripts/blink1-patt-tst.sh
blink1magenta-cyan-start-pattern.sh SAVES a boot pattern to a blink1 (USB-powered LED) device via blink1-tool (Mac).
Run without any parameter:
blink1blue-red-alternate-start-pattern.sh Work/break Pomodoro timer. Variables in this description are customizable via hacking the script:
homebrew, homebrew brightness package (Mac)
If you terminate this script in an interval of screen darkness, you will be left in darkness. DARKNESS, Batman. DARKNESS.
Execute this script without any parameter:
blinkBreakTimer.sh -- and let it run in the background as you work.
Creates a blank, transparent background png image of dimensions $1 (nn..Xnn..), via GraphicsMagick, named
imagemagick in your PATH, grep.
Run the script with one parameter:
Example command that will create a 5240x2620, blank, transparent png image named 5240x2620_blank.png:
newBlankIMG.sh 5240x2620 Example command that will create an opaque magenta image:
newBlankIMG.sh 5240x2620 f800fcff Converts all files of type $1 in a directory tree to type $2, via pandoc (via pandoc2doc.sh). See documentation in comments of that script.
Run with these parameters:
Example that will convert all files of type .md (markdown) to .html (static web page) :
pandoc2doc.sh md docx Overwrites a .hexplt file (pass it to the script as $1) with a hex-sorted copy of itself. You probably don't want to do this for color sorting though--RGB math sort may not align with human perception of what color seems near to what other color--you probably want to use and of various scripts that have ~`RGBhexColorSort` in their file name (see).
Run with one parameter, which is the file name of a .hexplt format file to sort this way, for example:
sort_hexplt.sh a_hexplt_file.hexplt Crawls all directories in the current path and runs palettesMarkdownGallery.sh (see) in every one.
For every image (of many types) in the current directory, resizes them to fit inside the smallest dimension of the smallest found image. The shrunk images are useful as bases for composites or comparisons (where otherwise those would operate on pixels in one image out of range of another).
Hack the script this calls (if you need to, to get a different formats list), then run this without any parameter:
gm_downsize_img_copies_to_smallest.sh Wrapper for youtube-dl which passes preferred CLI options to program before download URL $1
Run this script with one parameter, which is the URL to a video to download:
youtube-dl.sh URL-to-video Takes input file $1, sorts its lines, reduces it to unique entries, and prints that. Optionally writes that back to the file $1. To maintain order (or in other words to not sort), but eliminate duplicates, see getUniqueWords.sh.
Run with these parameters:
For example, to print unique lines from fileNameToSortAndDedup.txt, run
sortuniq.sh fileNameToSortAndDedup.txt To write the result to a new file, pipe it:
sortuniq.sh fileNameToSortAndDedup.txt > fileName_sorted_and_deduped.txt To filter fileNameToSortAndDedup.txt to unique lines and overwrite the original with that result, run:
sortuniq.sh fileNameToSortAndDedup.txt FLOURPESCENSE Renders any image source as a composition of flat primitive shapes, via a Go package named "primitive" (this is a wrapper for primitive). Output file names are the same as input only possibly in a different image format, but add _primitive_count_<count>_mode_<mode>__<8RandomChars> to the file name (extrapolate the meaning--it doesn't literally use angle brackets in the output file name). It adds random strings to the end of the file name, so that you can run it twice on the same input and get a different result (it is different every time!) and keep the original and the new alongside each other. Also, it overrides the (at this writing) default behavior of reducing the image by getting and passing the largest dimension to the -s switch.
Go, and the Go package named "primitive;" install it with:
go get -u github.com/fogleman/primitive --and the Go bin or wherever it installs said package in your $PATH.
Run with these parameters:
Example that will use input_image.png and make 100 rectangles, and render to a .png image:
primitive_render.sh input_image.png 100 5 png You can also just use the go package directly, as the basic usage is, well, basic:
primitive -i input.png -o output.png -n 100 -m 4 To directly use the go package, you can use the -o flag multiple times. This way you can save both a PNG and an SVG, for example.
https://gist.github.com/Everlag/8344fa7c9234900ba2cb851581c62599 re https://github.com/fogleman/primitive/issues/28 -- and see other things there
Concatenates all video files (default extension .mp4) in a directory into one output file. Source files must all be encoded with the same codec and settings. In contrast to some other scripts here, it makes the list of videos "on the fly"--it will do so with whatever sort the ls command outputs.
See "NOTE" echo at end of script (about super long path/file names).
ffmpeg, randomString.sh Run with these parameters:
$1 OPTIONAL. Extension of source video files to concatenate into one larger file. They must all be encoded the same way. If not provided, defaults to mp4.
$2 OPTIONAL. File name of list of files. The contents of this file must be in this format:
file 'file1.avi'
file 'file2.avi'
file 'anotherVideoFileName.avi'
file 'etcetera.avi'
If $2 is used, $1 is ignored (and can technically be anything in that case, like the word NULL). It may be desirable to use $2 if you want a custom ordering for the concatenation of the files. If you only use $1, whatever order results from the file list wildcard portion of the printf command in code (see) is the concatenation order.
Example with only parameter $1:
concatVideos.sh avi Example without any parameter, which would concatenate all mp4 files, because mp4 is the default if you don't provide a file type:
concatVideos.sh The result concatenated file name in that case is _mp4sConcatenated.mp4`.
Example with parameters $1 and $2:
concatVideos.sh YORGBLAF fadeSRCvideosList.txt This script sets an environment variable $concatenatedVideoFileName which will be set in a calling shell if you call this script this way:
source concatVideos.sh YORGBLAF fadeSRCvideosList.txt The purpose of setting that variable is to use it in other scripts that need to know the result encoded file name (to do things with it).
Force rebuilds the MacOS Spotlight index.
Run without any parameter:
rebuildMacSpotlightIndex.sh Runs a GraphicsMagick average operation against every possible file pair in the current directory to produce art from the average of pixels in the same position in two images. Can be really interesting with e.g. satellite photography of Earth (wilderness and/or civilization as seen from space), or two different abstract art works, or a satellite image and an abstract art work, or potentially any two different (or the same!) types of sources. NOTE that the commands will break on unsupported format files.
gm (GraphicsMagick)
Run without any parameters:
gm_average_all_img_pairs.sh Runs idiff (from openimageio toolset) against every possible file pair in the current directory to produce interesting diff art. Can be really interesting with e.g. satellite photography of Earth (wilderness and/or civilization as seen from space). NOTE that the commands will break on unsupported format files.
A 'Nixy environment, the openimageio tool set, images to diff, and printAllIMGfileNames.sh.
Run without any parameter:
idiff_all_img_pairs.sh IF SOURCE IMAGES are of various resolutions (not all the same), you may wish to first create copies of them all at matching resolutions via gm_downsize_img_copies_to_smallest.sh, and then run this script against the resulting /__smaller_img subfolder. Then run this script.
Converts a fountain format plain text file screenplay into a formatted PDF via a CLI tool (two options; the default one can be overriden with a second parameter). To join semantic linefeeds in a fountain file before converting it with this script, see fountain2fountain.sh. To use both fountain2fountain.sh and this script, see fountain2fountain2pdf.sh.
Wrap or Afterwriting CLI (depending on which you choose and which code line you uncomment for either, respectively), and various GNU core utilities.
Run with these parameters:
Example that uses the default renderer.
fountain2pdf.sh fountain-source-file.fountain Example that uses aftewriting:
fountain2pdf.sh fountain-source-file.fountain FLOREFLEF Splits all files in the current directory of type $1 into subdirectories by multiples of $2 (axe files into folders), with folder prefix name $3. If a IMGlistByMostSimilar.txt file (used by some scripts in _ebDev) is present, splits that into copies in the subfolders, these partitioned text file copies reflecting the files in the subfolders.
With this script in your PATH, run with these paramters:
_toEndFR_. If you want to use $4 and leave this as the default, pass the word DEFAULT for this parameter.Example command that will axe all files with the extension .hexplt into 80 files per folder, prefixing the numbered folder names with startN:
axeNfiles.sh hexplt 80 _startN_ Example command that will do the same thing but using the default folder prefix name, and randomly shuffling the list of files before sorting into folders:
axeNfiles.sh hexplt 80 DEFAULT FLORGULB If you use parameter $4 in a folder with an accompanying IMGlistByMostSimilar.txt, it will render the generated text file lists useless, because they won't reflect the random (shuffled and not matched) files in the folders.
Creates a .jpg (by default) file from an .svg file passed as parameter $1.
Run with these parameters:
For artwork publication tracking via annotated metadata. Retrieves and lists the EXIF ImageHistory section from the metadata prep field of that name, from all file names of a specific pattern, looking for a specific string, and writes the results to a reference file. See USAGE.
From a directory with file names of the pattern <anything>_EXPORTED_<anything>_MD_ADDS.txt (in the immediate folder and/or subdirectories), run this script:
listAllWorkLabels.sh The script will write the full path of all file names with that pattern in the file name and which contain the string "earthbound" (in an EXIF ImageHistory tag metadata prep. line) to __ALL_LABELED_WORKS.txt, then copy the ImageHistory metadata prep text from all those files to __ALL_WORK_LABELS.txt, for parsing and sorting by hand.
This expects UPPERCASE_LABELS to to be between the text -EXIF:ImageHistory= and . (the ImageHistory tag and a period, then space). If any intended labels are otherwise, they will not be found.
Compresses the video and audio from any video container (that ffmpeg will handle) with lossless codecs (utvideo and flac) into a new file of the same container format (e.g. .avi, .mp4). The new video will be named
Run with one parameter, which is the input video file name:
utVideoSound.sh inputVideo.avi Or for example:
utVideoSound.sh inputVideo.mp4 etc.
Adds blank sound to a video by re-muxing the original video stream (stream copy--so lossless copy) into the same container format, only with a new, silent 11000Hz aac audio track the duration of the clip. Blank sound will be aac encoded--hack this script if the source container isn't compatible. Needed e.g. for stupid TVs that complain if a media file has no sound track (instead of just ignoring it).
This replaces the original video without warning, with the new copy that has a silent sound track added.
ffmpeg Pass this script one parameter, which is a video file, e.g.:
addBlankSoundToVid.sh inputVideoFile.mp4 Renders a phrase to an image using every font file found in the current directory, with a caption that gives the font name. Useful for logo/font prototyping. Caption, point size, and image width are customizable via parameters; see USAGE. The render png is named after the source font file. Will not render to target file name that already exists.
ImageMagick and a directory full of .ttf and/or .otf fonts you wish to render a phrase from.
Run with these parameters:
Example that will render the phrase "COLOR GROWTH color growth" for every font file in the current directory:
previewFonts.sh "COLOR GROWTH" Example that will do the same but make every preview 650 px wide:
previewFonts.sh "COLOR GROWTH" 65 Example that will do the same but make the point size 16:
previewFonts.sh 65 \n), ImageMagick (which this script uses to render fonts) will insert a line break where you type that.Creates .jpg (by default) files from all .eps files in a directory tree, via ImageMagick. Creates 1280px jpg images by default. Also, does not overwrite files if the render target name exists (you must first delete the existing target file, then run this script to re-create it).
GraphicsMagick and ghostscript, both installed and in your PATH.
Run this script with these parameters:
If the result image has raster upscale (jaggy) artifacts, hack the -density parameter in this script with a higher number until that isn't a problem.
Takes a string parameter $1 and echoes a terminal-friendly string (for example for file names); Fixes Terminal Un-friendly Strings.
Run this script with one parameter surrounded either in double or single quotes. For example:
ftunStr.sh "Mountain@Landscape over] Downtown'' Provo UT" OR
ftunStr.sh 'Mountain@Landscape over]"" Downtown Provo UT'
To make use of the echoed result in another script by storing it in a variable, use command substitution:
terminalFriendlyString=$(./ftunStr.sh "Mountai'n Landscape#$ over Downtown Provo UT") Creates a markdown top-down image gallery from all image files in the current path (at this writing only png, jpg, gif, and tif are supported). Gallery file is README.md.
This script will overwrite any README.md gallery that already exists in the directory you run it from, without warning!
Run this script without any parameters:
imgs2MDgallery.sh (Omit the ./ if the directory which contains this script is in your PATH environment variable.)
Gets an image title from metadata prep ~MD_ADDS.txt files and runs WPmedia2gallery.sh with that title as a parameter.
Pass this script one parameter, which is a correctly populated ~MD_ADDS.txt metadata prep file name (which file this script will process) ; e.g.
MD_ADDS2markdownGallery.sh _EXPORTED_work_00001__2011-10-16-nearNovatek-IMG_0616-b-postP-layersMerged_MD_ADDS.txt Losslessly copies sound and video out of (virtually) any media file starting at $1 seconds for $2 duration (according to parameters passed to script), and also copies metadata and timestamp from source to excerpted copy.
ffmpeg, copyMetadataFromSourceFileToTarget.sh from _ebDev
Run this script with these parameters:
Example that would copy a 30 second sound and video clip starting at 12 seconds:
ffmpegLosslessVideoExcerpt.sh inputFile.mp4 12 30 To excerpt everything after the start point, put a stupidly high number as the "how long" (third) parameter. Maybe that's a kludge, but it works.
You (apparently) may also specify start and endpoint with ffmpeg like this:
ffmpeg -i inputFile -c copy -ss 00:09:23 -to 00:25:33 outputFile
Losslessly copies sound out of (virtually) any media file starting at $1 seconds for $2 duration (according to parameters passed to script)
Run with these parameters:
Example that would copy 30 seconds of sound starting at 12 seconds:
ffmpegLosslessSoundExcerpt.sh inputFile.m4a 12 30 To excerpt everything after the start point, put a stupidly high number as the "how long" (third) parameter. This is a stupid kludge but it works.
Runs gorisDownloadNmatches.sh against every (image) file of a given type in the current path.
Run this script with this parameter:
For example:
gorisDownloadNmatchesType.sh png Wrapper for goris Google Reverse Image Search CLI tool. Downloads N (from parameter $2) images matching or near matching file $1. See NOTES for details on where results and the original image are moved/saved to. SEE ALSO gorisDownloadNmatchesType.sh and the comments therein.
The API this relies on has a number of downloads limit per day (100, I read). I believe the error you'll see when this is hit is "panic: runtime error: invalid memory address or nil pointer dereference--" which tells nothing direct about hitting the limit. Also, even before you hit that limit, it may not download the full number of images you specify. Maybe Google throttles the API?
Run this script with these parameters:
Example that searches for imageFile.png and downloads the 6 top matches:
gorisDownloadNmatches.sh imageFile.png 6 Because the CLI tool this wraps downloads all matches to the current directory--which could clutter up your workspace fast--this script organizes search result downloads per the echo information at the end of this script.
Renders a sequence of image crossfades from a list (e.g. by next most similar image), by calling ffmpegCrossfadeIMGsToVideo.sh repeatedly.
a 'Nixy environment, sed, paste, tail, Perl, and the tools needed by ffmpegCrossfadeIMGsToVideo.sh.
See "DEPENDENCIES" first. Then run:
ffmpegCrossfadeIMGsToVideoFromFileList.sh For a recipe that uses this script, see next_most_similar_image_crossfade_anim.sh.
Cleans up the MacOS "Open with" menu.
Run without any parameter:
macOpenWithMenuCleanup.sh Runs hexplt2ppm.sh for every .hexplt file in the path from which this script is run (non-recursive).
Run this script, optionally with the following parameters:
Here is an example command that will create a palette image for every ppm file in the current directory, where each tile (in each palette image) is a square 250px wide, and tiles are rendered in random order, and the palette image is 5 columns wide and 6 rows down:
allhexplt2ppm.sh 250 foo 5 6 Runs renderHexPalette.sh for every .hexplt file in the path (non-recursive) from which this script is run. Result: all hex palette files in the current path are rendered. Also optionally recurses into subdirectories. Also has cooldown (no work) periods after every N renders.
Run with these parameters:
To render all palettes in the current directory, run the script without any argument:
renderAllHexPalettes.sh To recurse into all subdirectories and render all palettes in them, pass any parameter other than the word 'NULL' for $1:
renderAllHexPalettes.sh YORP To NOT recurse into subdirectories but also use additional parameters, pass the keyword NULL for $1, e.g.:
renderAllHexPalettes.sh NULL 250 NULL 5 Creates a markdown image listing (README.md) of all .png format palette files (rendered from .hexplt source files) in the current path.
Run this script without any parameters:
palettesMarkdownGallery.sh On Mac (at least) this script throws an error about test, and yet it still works as intended (the test causes a zero or nonzero return code).
Places a copy of every revision of a given file (from a git repo) to files named originalFileName.ext__ver_"nnn".txt. Files will appear in the same path(s?) as the file for which revisions are scanned. Updates extracted file time stamps with touch and /or exiftool to match git commit time.
With this script in your PATH, and from the root directory of a repo, run:
gitDumpAllFileVersions.sh relative/path/to/filename/filename.code This must be done from the repository root directory and you must give the full path to the file from that root directory.
If you have ridiculously long history with a file, and/or the file was ever renamed, you may want to first run:
git config diff.renameLimit 999999
I tried adding -S flag to the git log command and it seemed not to effect rename tracking at all. ? (re: https://stackoverflow.com/a/28064699/1397555 https://stackoverflow.com/a/5743887/1397555
Extracts image layers from a Photoshop file to numbered, individual images, in a subfolder named
SEE ALSO extractIMGlayersGM.sh OR TRY: Photoshop -> File menu -> Export -> Layers to files (and the alpha may only look good if you export to png-24)! This script exports layers in an image file (e.g. psd or tif source file) to png images.
Run with these parameters:
EXAMPLE COMMAND that will rip all layers from the file inFile.psd, and place them in a subdirectory named inFile_scenes:
extractIMGlayersIM.sh inFile.psd EXAMPLE COMMAND that will rip the 2nd layer from the file inFile.psd:
extractIMGlayersIM.sh inFile.psd 2 EXAMPLE COMMAND that will rip the topmost layer (scene) from the file inFile.psd:
extractIMGlayersIM.sh inFile.psd TOP It may work with various layered formats, for example also tif files:
extractIMGlayersIM.sh inFile.tif -- or perhaps any other supported layered file format.
A wrapper for markdown-styles (generate-md) to create an HTML publication from a source .md (Markdown) format file. Writes result files to a ./_publish_MD_tmp_qJt5d4bKCm folder.
This scripts deletes and recreates the ./_publish_MD_tmp_qJt5d4bKCm folder when run, without warning.
Nodejs with markdown-styles (generate-md) installed, and an .md-format file to convert.
Run this with one parameter, which is the input .md-format file to create an HTML publication from, e.g.:
publish_markdown_style.sh inputFile.md Resizes an image by nearest neighbor method via ffmpeg.
Run this script with three parameters:
For example:
ffmpeg2imgNN.sh input.jpg png 1920 Variant of data_bend_2PPMglitchArt.sh. Represents arbitrary bytes (from any file) as data where each byte becomes one of the components of an RGB color (hard-coded to result in blue to violet depending on the 0 to 255 value of the raw data which is made into a component of <VALUE 0 255>, where VALUE is the data value from the source data). See also all_data_bend_type2PPMglitchArt.sh, which will call this script against every file of a given time in a path.
a Unix environment including the od utility, and optionally IrfanView and img2imgNN.sh
Run with one parameter, which is a data source file name:
data_bend_2PPMglitchArt00padded.sh dataSource.file Makes glitch art from any data source by creating a ppm header approximating a defined image size (at this writing square) into which that data would fit; takes that image header and slaps raw copied hex value pairs (converted to decimal) into an RGB value array which composes the remainder of the PPM format file. The result may be converted to any other image format, apparently only by IrfanView (GraphicsMagick and NConvert choke on ppm files with hex values; IrfanView doesn't). SEE ALTERNATE script data_bend_2PPMglitchArt00padded.sh for a better data representation option.
a Unix environment including the od utility, and optionally IrfanView and img2imgNN.sh
Run with one parameter, which is the file name of the data source:
data_bend_2PPMglitchArt.sh dataSource.file You may be able to reliably reverse the process to recreate an original file a PPM was made from: all of the hex values for a source file are recorded in a resulting PPM via this script. In other words, this may be a way to obfuscate data (but note that the obfuscation is easily reversed).
Makes glitch art (via data bending) from any data source by creating a bmp header approximating a defined image size (at this writing square) into which that data would fit; takes that image header and slaps raw copied data from any source onto the end of it. It breaks the bmp spec and yet many image editors will display and convert the image (to non-broken "glitch" converted images) anyway.
GraphicsMagick, dd
Run with one parameter:
Exampe command:
data_bend_2BMPglitchArt.sh dataSource.file Indexes all text files in a directory tree with a file name of pattern <anything>_EXPORTED_<anything>_MD_ADDS.txt (case-sensitive) which contain an EXIF ImageHistory label (or tag or keyword) containing the string $1 (first parameter to this script); it also lists files that do not contain the label--and files that don't contain that label mean a work has yet to have history (is yet to be published e.g. at a web site, the way I use this script). See USAGE for further explanation.
Everything Search Engine CLI (and an install of Everything search engine tool), and therefore Windows, GNU CoreItil ports (probably from MSYS2) including sed
Run from the root of a directory tree with so many so $1 -patterned text files you wish to index; e.g.:
indexWorksByLabel.sh earthbound The script will write the full path of all file names with the pattern <anything>_EXPORTED_<anything>_MD_ADDS.txt which contain the string "earthbound" (in an EXIF ImageHistory tag metadata prep. line) to __LABEL_MATCHED_WORKS.txt, BY OVERWRITE (the file contents will be replaced). Such file names which do not contain that string will be written to __LABEL_NOT_MATCHED_WORKS.txt, also by overwrite.
These files are reference for publishing my art work (determining what to publish next).
-EXIF:ImageHistory=".*publication.* in all so-named text files it finds.<anything>_MD_ADDS.txt files associated with artwork which function as a database of what has been published where. I have a file named MD_ADDS_publication indexing labels tags keywords.txt among my own files which lists those. Maybe it should be a public file? A different version of it, I think, used to be . . .Outputs a .jpg image frame from a given percent (time) of an input video file. Output image is video_frame.jpg at this writing (plan: name it after the input video).
Run with these parameters:
Example:
ffmpegRipVideoFrameSS.sh inputVideo.mp4 .86 Copies the whole input video to a temp file; inefficient.
Combines all input image files of type $1 (parameter 1 to script) into a layered output image file. Suggested options: kra, ora, or psd; ora (works with Krita) recommended.
GraphicsMagick and Krita, both in your $PATH.
Run this script with these parameters:
Example run command:
allIMGtypeToLayeredIMG.sh tif ora Wipes Xcode derived data directory (Mac), which may be necessary if
Run without any parameters:
wipeXCodeDerivedData.sh krita CLI wrapper to convert from a krita source image to another destination image format (you might call this e.g. effectively kra2png).
An install of krita http://krita.org in your PATH.
Run this script with these parameters:
For example:
krita2IMG.sh input.kra png Renames many image, sound and video files (of many supported types, and in the current directory) after dateTimeOriginal and createDate metadata. As this is an irreversible process (unless you keep backups), it asks you to enter a password, which it presents to you, to continue.
ExifTool
Run from a directory with media files you wish to so rename, e.g.:
renameByMetadata.sh OR OPTIONALLY, to bypass the password check and rename all files by metadata without warning, run with one parameter, which is the word NORTHERP:
renameByMetadata.sh NORTHERP You can view all timestamp metadata in a file with this command; you would replace <inputFileName.file> with an actual source file name you want to get the metadata for:
exiftool -time:all -g1 -a -s <inputFileName.file>
This is a list of all supported exiftool file types for which tags can be written, obtained via the command exiftool -listwf: 360 3G2 3GP 3GP2 3GPP AAX AI AIT APNG ARQ ARW AVIF CIFF CR2 CR3 CRM CRW CS1 DCP DNG DR4 DVB EPS EPS2 EPS3 EPSF ERF EXIF EXV F4A F4B F4P F4V FFF FLIF GIF GPR HDP HEIC HEIF HIF ICC ICM IIQ IND INDD INDT INSP J2K JNG JP2 JPE JPEG JPF JPG JPM JPS JPX JXL JXR LRV M4A M4B M4P M4V MEF MIE MNG MOS MOV MP4 MPO MQV MRW NEF NKSC NRW ORF ORI PBM PDF PEF PGM PNG PPM PS PS2 PS3 PSB PSD PSDT QT RAF RAW RW2 RWL SR2 SRW THM TIF TIFF VRD WDP X3F XMP
ffmpeg wrapper that crossfades two videos with custom fade parameters. Result file name is <input_video_one_base_file_name>_xFade_<input_video_two_base_file_name>.mp4.
Run with these parameters:
Example that will crossfade input1.mp4 to input2.mp4, with a crossfade of 2.3 seconds, which begins at 7.6 seconds into the video:
ffmpegCrossFadeVideos.sh input1.mp4 input2.mp4 2.3 7.6 Makes a video of a random color accompanied by a random sound pitch. Because you might want to drive someone insane, or because what is life and what is art and why are you doing this and what is life and what is art and why are you doing this.
sox CLI sound utility
Run with one optional parameter, which is a decimal duration in seconds which will be the duration of the generated video, e.g.:
RNDcolorAndPitchVid.sh 0.68 If you pass no parameter, it defaults to a hard-coded value.
Disables and even deletes contemptible or useless Windows services. This script is very shotgun "blast everything I even slightly don't like," including things you might want to keep around.
May break essential system functionality or services that programs rely on. Use at your own risk.
Read the NOTES comment.
Don't use this script unless you're very sure that no harm or unwanted operations will come to your operating system or programs if you use it. If you're sure of that, then: from a cmd prompt with administrative privileges, and with paexec in your path:
fund and extract one NSUDO.bat / utility, e.g. https://nsudo.m2team.org/en-us/
run that batch to get that utility
point it to MSYS2 e.g. "C:\msys64\msys2_shell.cmd -msys"
From that MSYS2 super-elavated terminal, run this script (you may need to cd to the directory with it first) :
fryStupidWindowsServices.sh
Possible alternate route to merely disable unwanted services:
OR from that NT/System Authority-priviledge prompt run:
sc delete "service name" As of Aug. 2018 (or earlier), Windows malignantly re-enables windows update and the commands here that seek to disable that don't work--services that switch windows update back on cannot be disabled.
re: https://answers.microsoft.com/en-us/windows/forum/windows_10-other_settings/windows-10-windows-update-keeps-turning-it-self
However, if you find one NSUDO tool and run MSYS2 from it as TrustedInstaller with "Enable all Privileges," this script will disable those. You have to be a super-duper admin destroy user (as noted above) like that.
Example service control commands:
sc config "AeLookupSvc" start= demand
sc config "NgcSvc" start= disabled Installs all the nodejs (npm / node) modules I commonly use.
Run without any parameters:
install_global_node_modules.sh To get a list of npm packages (without all their dependencies) and paste it into this list, run:
npm list -g --depth=0 > installedNPMpackages.txt Installs all the brew packages I commonly use.
Run without any parameters:
installUsedBrewPackages.sh To get a list of all homebrew installed packages to back up here, run:
brew list -1 > installedBrewPackages.txt Also, to uninstall every brew package but keep brew, run; re: https://github.com/Homebrew/legacy-homebrew/issues/48792
brew list -1 | xargs brew rm It's my preference to put executables from the installed path of coreutils before Mac's built-in tools of the same name, re: https://formulae.brew.sh/formula/coreutils -- which I accomplish in macDevSetup.sh via a printf command appending the following to ~/.bash_profile:
PATH="$(brew --prefix)/opt/coreutils/libexec/gnubin:$PATH" And:
PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH" ALSO, see pyenvFrameworkInstallMac.sh.
Re-sorts result list from imgsGetSimilar.sh according to whatever criteria you alter on the sort code line in this script (if the original results weren't to your liking).
Examine the sort commands near the start of this script, uncomment the one you want, comment out the others, and run this script:
re_sort_imgsMostSimilar.sh Converts (almost?) any camera raw format image file into a .tif file via dcraw. See NOTES for a command to quickly extract any embedded thumbnails.
dcraw Run this script with one parameter, which is a raw image filename in your PATH. e.g.:
dcraw2tif.sh rawImageFileName.cr2 See the comment about the -W option in the code.
To quickly rip the embedded jpegs (if there be any) out of all images, don't even use this script, run:
dcraw -e *.CR2
An interesting thing is to see raw sensor values in white scale:
dcraw -D -T inputFile.cr2
See documentation comments throughout code for details on dcraw options, etc.
I may have read somewhere that if you can find a camera profile for your camera and use it with dcraw, you'll get output as good as or better than manufacturer utilities. That may be something to explore.
Takes two input images and creates a video crossfading between them, via ffmpeg, with fade duration and padding options via parameters to this script.
ffmpeg, a nixy' environment
run this script with three parameters:
Example that creates a video of a 7-second crossfade from one image to another, with 4.36 seconds padding before and after:
ffmpegCrossfadeIMGsToVideo.sh inputImageOne.png inputImageTwo.png 7 4.36 You may wish to hack global variables right after the CODE comment per your wants.
This script will not overwrite pre-existing render targets. You may therefore interrupt and re-run it to stop and continue interrupted render series.
If this script is called this way from another script via the source command, like this (assuming this script is in your PATH) :
source ffmpegCrossfadeIMGsToVideo.sh
-- then the variable which this script sets, named $targetRenderFile will persist in the shell after this script terminates, for a calling script to make use of. The script ffmpegCrossfadeIMGsToVideoFromFileList.sh does this.
Calls dcraw2tif.sh for every .CR2 format file in the current directory (thereby making .tif format conversions of them).
Performs administrative setup per my preferences for MacOS development etc.
Examine every line and uncomment what you want, and comment out what you don't want. If you don't know what every line does, don't use this script. Then run this without any parameters:
macDevSetup.sh Then follow the prompt to really AKTULLY do things if you want (requires a password it tells you).
If it's your preference, revert from newer MacOS Zsh shell to bash with this command:
chsh -s /bin/bash To go to Zsh again, run:
chsh -s /bin/zshcd Lists all globally installed nodejs (npm / node) modules.
Run without any parameter:
list_global_node_modules.sh Commands to install wanted tools and custom configuration for a Cygwin install.
Maybe don't use this. I have come to far prefer MSYS2 over Cygwin. See MSYS2setup.sh. But if you use this, run it from the directory with the installer, with this script and the other files it needs (CYGWIN_fstab, my.minttyrc.settings.txt) in your PATH, or copy the Cygwin installer to this directory before executing this script. Comment out the line for the Cygwin installer version you don't want to use.
Then open the Cygwin prompt, cd to this path, and run the script without any parameter:
cygwinSetup.sh Produces pseudo-random data which for all anyone knows could be called true random data if you begin with pure random entropy files like jpeg photographs. Reference: https://crypto.stackexchange.com/a/43121
Run this script only against a copy of data or on expendable data--it will destroy all original data in the folder from which it was run! It moves generated PRND data into the ../_final_TRND_archive folder.
xxd, rehash, and unless rehash is available on other platforms, Windows. On Cygwin install it via apt-cyg install hxd.
Run with one optional parameter, which is the block size to split the concatenated data into before hashing, e.g.:
PRNDfromEntropicData.sh 780 -- if the optional parameter is not chosen it will pseudo-randomly choose a split block size between 464 and 1247.
This script creates a .dat file by 1) concatenating all data in a directory into one file, moving that new file out of the way, deleting all files in the directory, moving the concatenated file back, and 2) cutting the file (binary split) into files of a size of bytes pseudo-randomly chosen between 242-512 bytes, extracting noise from all of them via non-cryptographic hashing (and collating the hashes into one hex string and interpreting that as binary values written to a new random data file (.dat). It then moves the new random data file into an archive folder and concatenates all the split files into one file which may be "recycled" with this same "chaos machine" process.
Pseudorandomly picks a number in a range of bytes corresponding to byte size of file $1 (parameter $1 passed to script), and prints it. This number is intended to be used with e.g. dd for random byte sampling (e.g. from a give byte start position), for data bending/glitch art.
getRNDinFileBytesRange.sh inputFile.png To use from another script to assign the random byte index to a variable:
rndByteIDX=./getRNDinFileBytesRange.sh inputFile.png
Makes an expanded bytes copy of an input file (parameter 1), with each byte repeated N (parameter 2) times to an output file named like sourceBaseName__dataStretched__.dat. Intended for data bending / glitching e.g. importing the result file as raw uLaw data in Audacity audio to "hear" it in terms of a pattern more clearly. Could work for stagonography also, but only if the reversed process knows that the last copied bytes are copies of fewer bytes if the number of source bytes weren't a multiple of the second parameter.
Run this script with the below parameters:
For example, suppose you have a source file named data.dat, with these hexadecimal values for data (two hex characters per byte) :
AB CD EF 00 11 22 If you run:
data_bend_repeatXsourceBytesYtimes.sh data.dat 3 It gives this result (again shown as hexadecimal) in __dataStretched__sourceBaseName.dat:
AB CD EF AB CD EF AB CD EF 00 11 22 00 11 22 00 11 22 The pattern is that every group of bytes size $2 is repeated $2 times.
Even for relatively small amounts of data, this can be SLOOOOOOW.
A way to get some data for this for .bmp -> .wav data bending purposes, is to create a one-line many-column bitmap with this command:
colorsGridFromRNDorList.sh 70 1 1 rainbowHexColorsByMyEye.hexplt
Then in Photoshop convert the resulting .ppm to a .bmp
Then split it into a header and data file with these commands, that split the file into a 54-byte header.dat and the data remaining after that to data.dat:
dd bs=54 ibs=54 count=1 if=in.bmp of=header.dat dd ibs=54 skip=1 if=in.bmp of=data.dat
Deduplicates listings in bash history.
Run without any parameters:
cleanupBashHistory.sh ~/.bash_history to exist and contain bash history.Get N random bytes (from parameter $1) using openssl.
openssl
To write N random bytes to a file via this script, run it this way; here, 512 can be changed to any number of bytes you wish to generate:
openSSLrnd.sh 512 > rnd.dat From a list of image filenames suitable for ffmpeg (which file names must be in the current folder), creates numbered hardlinks in a subdirectory, from which an animation may be made, e.g. via ffmpegAnimFromFileList.sh. SEE ALSO mkNumberedCopiesFromFileList.sh if you prefer to make copies of the files instead of junctions (though copies may be slower to make).
IMGlistByMostSimilar.txt, as prepared by imgsGetSimilar.sh and/or re_sort_imgsMostSimilar.sh.
FIRST, prepare a list of images, for example named IMGlistByMostSimilar.txt, as prepared by imgsGetSimilar.sh and/or re_sort_imgsMostSimilar.sh. Or manually prepare your own list with any other file name. The list must have the following layout or format:
file '263.jpg'
file '363.jpg'
file '064.jpg'
file '145.jpg' This format is suitable for ffmpeg.
Then, run with these parameters:
Example run with a file list parameter:
mkNumberedLinksFromFileList.sh customImageListForAnimation.txt If such a file name is not provided as the first parameter, a file list must be present in the same directory you run this script from, and the file list must be named IMGlistByMostSimilar.txt.
To run the script without the optional first parameter, you would run:
mkNumberedLinksFromFileList.sh The script will scan the list and make numbered junctions (file names which are junctions) in a subdirectory named _temp_numbered. This is preparation for other scripts which must operate on numbered files.
From a list of image filenames suitable for ffmpeg (which file names must be in the current folder), creates numbered copies of those files in a subdirectory, from which an animation may be made, e.g. via ffmpegAnimFromFileList.sh. SEE ALSO mkNumberedLinksFromFileList.sh if you prefer to make junctions (links) instead of file copies (junctions may be faster to create).
IMGlistByMostSimilar.txt, as prepared by imgsGetSimilar.sh and/or re_sort_imgsMostSimilar.sh.
FIRST, prepare a list of images, for example named IMGlistByMostSimilar.txt, as prepared by imgsGetSimilar.sh and/or re_sort_imgsMostSimilar.sh. Or manually prepare your own list with any other file name. The list must have the following layout or format:
file '263.jpg'
file '363.jpg'
file '064.jpg'
file '145.jpg' This format is suitable for ffmpeg.
Run with one OPTIONAL parameter, which is the file name of any such list you have prepared.
Example run with a file list parameter:
mkNumberedLinksFromFileList.sh customImageListForAnimation.txt If such a file name is not provided as the first parameter, a file list must be present in the same directory you run this script from, and the file list must be named IMGlistByMostSimilar.txt.
To run the script without the optional first parameter, you would run:
mkNumberedLinksFromFileList.sh The script will scan the list and make numbered file copies in a subdirectory named _temp_numbered. This is preparation for other scripts which must operate on numbered files.
Calls glitchThisFile.sh repeatedly, to create glitched versions of all files of a given extension in the directory from which this script is run, producing N glitched image file variants for each file, output to a /_glitched folder. See USAGE for options. Written specifically for the purpose of deliberately making glitch art out of e.g. .jpg files, but it may produce "good" results for a variety of file formats. OR, with some code change, runs BM.exe (Byte Molester, a free tool).
Run with these parameters:
Example command that will create 10 glitched versions of all jpg images in the current directory, corrupting 30 percent of their data:
glitchAllFilesOfTypeNtimes.sh jpg 10 30 Uploads an image to instagram via a reverse-engineered API script call.
~/instagramPassword.txt. (I know, this is insecure, and I would like to find a way to make it more secure. Make the instagram password a hash of a very secure password, then prompt to type that password, and have the script hash it, so it is not stored in readable format?)After composer is installed and you have reloaded the PATH (e.g. by logging off and on), install Instagram-API via this DOS terminal command:
composer require mgp25/instagram-php At this writing, this script has been untested and not used for some time, and it may be broken after some blind tweaks were made to it for better array creation / command substitution. I know, fixing something I don't know is broken, and not testing it. :| ALSO, at this writing it is coded for Windows only (it runs .bat scripts, despite some untested code toward making it windows/Unix alternatly compatible).
Run with the following parameters:
Example run command:
call_instagramUploadPhotoParametersPHP.sh earthbound.io ./_EXPORTED_M_variantWork_00099_FFsideToside_v02_PZ-8280x.jpg ~/instagramPassword.txt./examples subdir of the Instagram-API repository must be in your PATH, as this script searches for one file UploadPhotoParameters.php in your path, and cds into that directoryWrapper that renders any text within quote marks to a randomly named audio file. Windows-only. For MacOS, you can run this:
say "blorf blefl horple" cat inputFile.txt | say OR (better yet) :
curl https://earthbound.io/data/gibberish-artist-statements/index.php?gib=florf | say Balaboka CLI (balcon.exe) / (windows text-to-speech engine tool) installed and in your PATH and MSYS2 or other 'nix environment on Windows.
Run this script with one parameter, which is any text to render to a .wav file (text encased in "quote marks"); e.g.:
balabokaTextToSpeechWav.sh "blarpnoi hoi hoi" These commands don't use this script, they are reference:
Grab a fake artist statement from the "Gibberish Artist Statement Dispenser" and say it out loud:
curl https://earthbound.io/data/gibberish-artist-statements/index.php?gib=florf > gib.txt
./balcon -f gib.txt Wrapper that runs autotrace.exe with custom parameters to retrieve (or attempt to retrieve) centerline paths from raster art of e.g. lines. Result will be found as <input_file_base_name>_centerline.svg.
Run with these parameters:
Example:
autotraceCenterline.sh input.bmp Prints $1 randomly generated hex strings of length $2 (default values used if no parameters provided).
Run with these parameters:
For example, to print 14 hex strings, each 42 characters long, use:
randomHexString.sh 14 42 To use default settings, omit any or all parameters, e.g.
randomHexString.sh 8 Or:
randomHexString.sh Wipes all the silly .DS_Store metadata files from the MacOS drive. It can take forever. SEE ALSO macDevSetup.sh to set a preference to not write .DS_Store metadata files on network drives.
Run without any parameter:
MacKillDSstoreFiles.sh Creates one or more .ppm (plain text file bitmap format) images which is W x H pixels of randomly generated colors OR colors randomly selected from a .hexplt format list of colors. Generates $3 such images. See USAGE for script parameters and examples.
Pass this script the following parameters:
NOTE that if you provide a source list of colors ($4), but numbers too small in parameters $1 and $2, it will not use all colors from the list (as it will generate tiles against only part of the list).
AFTER RUNNING this script you may wish to run e.g.:
imgs2imgsNN.sh ppm png 4280 4280 -- see the comments in imgs2imgsNN.sh for details.
Generate 3 files of randomly generated colors in a 4x2 grid:
colorsGridFromRNDorList.sh 4 2 1 Generate one hundred and seventy 16x9 pixel files of colors picked randomly from the color hex code list file rainbowHexColorsByMyEye.txt:
colorsGridFromRNDorList.sh 16 9 170 rainbowHexColorsByMyEye.txt The same as the previous command, but reading colors from the list sequentially:
colorsGridFromRNDorList.sh 16 9 170 rainbowHexColorsByMyEye.txt foo Overwrites all images of a given type $1 (in the directory you run this script from) with a text render of an arbitrary phrase and a number count of which image in the list has been written to.
GraphicsMagick, ghostscript
call this script from the terminal to this way destroy so many images e.g.:
imgs_of_numbers.sh png "SET A" To have global use of it, copy this script to somewhere in your $PATH, or make such a path.
Takes a list of hex color codes, one per line, and renders a PNG image composed of tiles of those colors (a palette image), via ImageMagick. This script is inefficient; there are probably much faster ways to make a palette image in the structure/format made by this script (it creates color tile images and then montages them), but at this writing, this script is what I have.
~/palettesRootDir.txt (in your home folder) which contains one line, which is a Unix-style path to the folder where you keep hex palette (.hexplt) files. If this file is not found, the script searches for palette files in the current directory.Run this script with the following parameters:
.hexplt format, which is a list of RGB colors expressed as hexadecimal (hex color codes), one color per line. If this file is in the directory you run this script from, it will be used. If the file is not in the current directory, it may be anywhere in a directory tree in a path given in a file ~/palettesRootDir.txt, and the script will find the palette in that directory tree and render from it.EXAMPLE COMMAND; create a palette image from the hex color list RGB_combos_of_255_127_and_0_repetition_allowed.hexplt, where each tile is a square 250px wide, the palette is 5 columns wide and 6 rows down, and tiles in the palette are rendered in random order:
renderHexPalette.sh RGB_combos_of_255_127_and_0_repetition_allowed.hexplt 250 foo 5 6 ANOTHER EXAMPLE COMMAND; create a palette image from tigerDogRabbit_many_shades.hexplt, with each tile 300 pixels wide, no shuffling, the script deciding how many across and down to make the tiles:
renderHexPalette.sh tigerDogRabbit_many_shades.hexplt 300 tigerDogRabbit_many_shades.hexplt ANOTHER EXAMPLE COMMAND; use the same palette and let the script use all defaults, including any number of tiles (columns) accross and down specified in the source hexplt:
renderHexPalette.sh tigerDogRabbit_many_shades.hexplt This script will work with many kinds of other information present in a .hexplt source file other than RGB hex codes. You can probably have any other arbitrary text, anywhere in the file, including on the same line as RGB hex codes, and it will extract and use only the RGB hex code information. However, no kinds of comments (like # or // at the start of lines) are supported.
Source hexplt files may contain syntax to define the desired number of columns and rows to render them with. The syntax is to write the word "columns" followed by a number on any line of the file, and optionally also the word "rows" followed by a number on any line of the file, like this:
columns 7 rows 8
-- or like this:
#D29B7D columns 7, rows 8 All that matters is that the word 'columns' appears followed by a number. You can specify columns only, and this script will figure out the needed number of rows. You can also specify rows (in which case the syntax is the keyword 'rows' followed by a number), and the script will use that number of rows, with the same conditions as for the number of tiles (rows) down parameters to this script.
allRGBhexColorSort~ scripts may be able to sort colors in palettes (it may not work if the columns and rows are specified on their own line).Makes a (non-standard?) teensy ppm palette image (one pixel per color) from a hex palette .hexplt source file. Result is (somewhat?) usable as a basis for a scaled up palette image via e.g.:
imgs2imgsNN.sh ppm png 640
NOTE first that you may wish to use hexplt2ppm.sh instead of this; re the KNOWN ISSUES.
If you opt to use this script, run it with the following parameters:
EXAMPLE COMMAND to create a palette image from the hex color list RGB_combos_of_255_127_and_0_repetition_allowed.hexplt, where each tile is a square 250px wide, the palette image is 5 columns wide and 6 rows down, and has squares in the palette rendered in random order:
hexplt2ppmHEX.sh RGB_combos_of_255_127_and_0_repetition_allowed.hexplt 250 foo 5 6 Makes a teensy ppm palette image (one pixel per color) from a hex palette .hexplt source file. Result is usable as a basis for a scaled up palette image; see NOTES
Run this script with the following parameters:
For example, to create a tiny ppm palette 5 tiles across and 40 tiles down from colors in the list RAHfavoriteColorsHex.hexplt, run:
hexplt2ppm.sh RAHfavoriteColorsHex.hexplt 5 40 To create a tiny ppm image with columns and rows calculated by the script from that same palette, run:
hexplt2ppm.sh RAHfavoriteColorsHex.hexplt To use another script to upscale the image to a palette by nearest neighbor (hard edge) method to a png image, run e.g.:
img2imgNN.sh source.ppm png 640 480 Gets color palettes (number of colors configurable via parameter) from an image via color-thief-jimp-pallete.js and ImageMagick, parses and combines them into one palette, and produces a palette image from the result.
nodejsImageMagicknodejs package installed globally: npm install -g jimpnodejs package installed globally: npm install -g color-thief-jimpRun this script with these parameters:
Example:
getHybridPalette.sh inputImage.jpg 15 Calls img2dng.sh for every file of type $1 in the current directory. But that script is DEPRECATED; the functional code of it is commented out and you'll accomplish nothing by calling that script from this one unless you uncomment the functional code in that one.
Call with on parameter, which is:
$1 file extension for which you wish to pass every file of that type in this directory to img2dng.sh. For example:
imgType2dng.sh CR2
-- will cause this script to call img2dng.sh once for every file of type .CR2 in the current directory.
For all of many image types in the current directory, creates text files of simplified metadata information, named after the image.
Hack the script this calls (if you need to, to get a different formats list), then run this without any parameter:
exportIMGsMetadataSimple.sh Converts all images of type $1 in the current directory to type $2, by repeated calls of img2img.sh.
For svgs, use SVG2img.sh or allsvg2img.sh, not this. This will do very crummy upscaling of vector images, post-resterization.
Run this script with these parameters::
Example that will convert all png images in the current directory to jpgs:
imgs2imgs.sh png jpg Rips all glyphs out of any TrueType (~`.ttf) font, saving each glyph into a vector file (epsorsvg` are possible, and maybe other formats).
FontForge and ImageMagick, both in your PATH.
Run with these parameters:
For example:
ripGlyphsFontForge.sh inFontFile.ttf eps A way to get FontForge in your PATH is to run FontForge-console.bat from the install directory of FontForge. Then, run commands to get into the directory you want to work in and run this script:
cd /path/to/working/directory
path/to/ripGlyphsFontForge.sh inFontFile.tff eps -- where inFontFile.ttf is the font file name to extract glyphs from, and eps is the output format for ripped glyphs.
Also, ripGlyphs.pe and the font file must both be in your PATH and/or working directory. Maybe. I just copy the files I want to work on into the FontForge directory temporarily, then copy out the results and clean up.
Wrapper for a command structure to launch a subshell. Launches whatever program you pass as parameter $1, and returns to the shell (and does not wait on the program), and the program does not terminate when you exit the shell.
Run with one parameter, which is the file name of a program or script (presumed to be in your PATH) to so execute. For example, to launch web_post_color_growth.sh in a process independent of the calling shell, run:
subshellLaunch.sh web_post_color_growth.sh Rips all basic Latin Unicode glyphs out of a .ttf font and generates individual ~.eps (PostScript) files of them.
ttf2eps from FontForge, which (I think) must be manually compiled for your system. you may compile it via ttf2eps.c, included in this repo. I have a win32-or-64 binary of it in _ebSuperBin.
Run with one parameter, which is the name of a true-type font file in the same directory from which you call this script. For example:
ripGlyphsTTF2eps.sh NotoSansMono-Black.ttf To extract all glyphs you don't need this script; just run:
ttf2eps -all <ttfFileNameInYourDirectory> You may get fewer or no errors (ripping from ttf to eps glyphs, or in subsequent eps to png etc. conversion) if you load a font in FontForge, then export it to a new .ttf font. That is also a path to extracting glyphs from fonts in formats originally other than .ttf.
Runs svgo_optimize.sh for every .svg file in the current directory. See comments in that script.
Run without any parameter:
svgo_optimize_all.sh Optimizes an svg input file (writing the result to <original_file_base_name>_opt.svg) including color code conversion suited for random recoloring via SVGrandomColorReplace.sh.
A nodejs install with the svgo (svgomg) package installed.
First examine the file .svgo.yml in this distribution, and if you wish to, copy it over the .svgo.yml file that comes with svgo. Among other things it preserves path IDs and long hex color form. UPDATE: that will need to change; the newest version of SVGO doesn't use yaml config anymore, it uses .js. For now, you're forced to either use the default config or update the yaml to the newest js and figure out how/where to copy that/use that. RE: https://github.com/svg/svgo/releases/tag/v2.0.0
Run this script with one parameter $1 (required), being the name of the svg file for which you want an ~_opt.svg file produced in the same directory; e.g.:
svgo_optimize.sh inputFile.svg
Cygwin. I've at times found that if I copy and paste the printed command to a cmd prompt, it works OK . . . except the result displays wonky in Internet Explorer and inkscape.Converts all video files of type $1 (in the current directory) to type $2, with default crf (constant rate factor or quality) 13 (quite high quality). Conversion may be to the same type, as the target is named after the original but adds "_converted" to the file name. Also copies metadata from source to target and updates target time stamp to match media creation, via another script.
ffmpeg, ExifTool, copyMetadataFromSourceFileToTarget.sh
Run with these parameters:
Example that will re-encode all files with the extension .mov to .mp4 files:
allVid2vid.sh mov mp4 SEE ALSO the "ADDITIONAL PARAMETERS" comment section.
Via ExifTool, wipes ALL image or video metadata (of all known types) from file $1 (parameter 1), via overwrite of original file. A known password prompt prevents from casual or accidental run of this permanent metadata-destroying operation.
Pass this script one parameter, which is a file name to wipe the metadata from in-place, for example:
wipeMetadata.sh inputFile.mp4 Do this only on data for which you have a backup! If something goes wrong with this, it can be a permanent kablooey for the affected files.
To wipe all metadata from all supported file types, pass $1 as . (meaning '.' or just a dot).
If you pass a second parameter which is the word 'YALBLOR', it will wipe metadata (from one or all files (one filename as $1 or '.' for all files) without asking.
Losslessly removes sound from all videos of type $1 (parameter 1), renaming the previous version of the file to
Run with one parameter, which is the extension of videos you wish to strip of sound, e.g.:
stripSoundAllVideosType.sh Lossy transcodes all video files in the current directory (and optionally all subdirectories) into mp4s compatible with Sony Vegas 12 (and higher, and maybe lower versions also). The "Vcompat" in this script name means "Vegas compatible," or compatible with Sony (V)egas video editing software.
ffmpeg To convert all files in the current directory, but not subdirectories, run without any paramter:
allVidsType2VcompatMP4.sh To convert all files in the current directory and all subdirectories, run with any parameter (for example the word 'BROGNALF':
allVidsType2VcompatMP4.sh BROGNALF At one point this script worked for my purposes by only encoding in x264, I think. Some time after that it stopped working, and encoding to yuv420p pixel format fixed it.
Generates random hex color schemes of file format .hexplt (randomly named), which are plain text files with one hex color per line.
Run with these parameters:
Example that will generate 10 random palette files with 6 colors each:
NrandomHexColorSchemes.sh 10 6 Example that will generate 12 random palette files, and the script will randomly decide how many colors in each:
NrandomHexColorSchemes.sh 12 Prints text file $2, modified, with all words in text file $1 removed from it. Another way of saying that is: prints all words in $2 that don't appear in $1. Both files must have one word per line.
Run with these parameters:
For example:
filterExcludedWords.sh filterExcludeWords.txt fileToFilterWordsFrom.txt The source text files are expected to have one word per line. This will not work the way you might want with paragraphs.
To capture results to a new file, use a redirect operator, like this:
filterExcludeWords.sh filterExcludeWords.txt fileToFilterWordsFrom.txt > filteredWords.txt
For correcting date stamps in files in Windows file systems that get into inconsistent or outright wrong states by backups, restores or other file operations. Via ExifTool, scans the creation date, modification date, access date, metadata Create Data, and Metadata Date/time stamps (the latter two for image etc. files), then sets the Windows file creation date/time (possibly unique to windows--not in 'nix file systems) to the earliest of these, and the file modification time to the newest of these.
exiftool, sed, head, tail, binarez_touch http://www.binarez.com/touch_dot_exe/ (a copy is in my _ebSuperBin repo).
Run with the name of a file to so modify the timestamps of it, e.g.:
toOldestWindowsDateTime.sh image.jpg OR e.g.:
toOldestWindowsDateTime.sh source_file.txt To do this for every file (regardless of file type) in the current directory, see allToOldestWindowsDateTime.sh.
Creates a subdirectory of hardlinks to all files of type $1 in the current directory, the hardlinks being numbered file names (digestible e.g. by image processing scripts to create animations).
Run this script with these parameters:
Example that will make hardlinks to all png images:
mkNumberedLinks.sh png Example that will do that and randomly shuffle the image list before hardlink creation:
mkNumberedLinks.sh png BLAERFNOR Renames all folders in the current directory (but not subfolders; non-recursive) with random characters to length $1.
Run with one parameter:
Example that will will rename all folders (non-recursive) with 34 random characters:
allRandomFolderNames.sh 34 Resizes all images of type T (via parameter $1) in the current path, by nearest-neighbor method, to target format F ($2), at size A x B ($3 x $4). (Nearest neighbor method will keep hard edges, or look "pixelated.") Runs img2imgNN.sh repeatedly to do this.
This script uses the same parameters as img2imgNN.sh, EXCEPT that parameter $1 is a file type instead of a specific file. All files of type $1 will be passed to imgs2imgsNN.sh:
Example command:
imgs2imgsnn.sh ppm png 640 OR, to force a given x by y dimension for a ppm:
imgs2imgsNN.sh ppm png 640 480 Produces list of images in the current directory arranged by next most similar. Compares all images in a directory. For the first image, it lists which image is most similar to it, then does the same for the second, then third image, and on until the end of the image list. The result is a list of images where every image is adjacent to the two images which are most similar to it. See NOTES for potential uses. It may end up that sort order is not strict; there may be some some randomization in sorting so that most nearly-identical images are not always clumped together with least similar images toward the head or tail of the list. I have not re-examined this since coding it and did not document that.
printAllIMGfileNames.sh, GraphicsMagick, image files in a directory to work on, and bash / GNU utilities
Run with these parameters:
For example, to compare all png format images in the current directory, run:
imgsGetSimilar.sh png To compare all (supported) image formats in the current directory, run:
imgsGetSimilar.sh Find all corrupt images of type $1 in the current directory, and move them into an /_irrecoverable subdirectory for examination.
GraphicsMagick (gm identify)
Run this script with one parameter, which is an image file extension type without any . in it. For example:
identifyCorruptIMGs.sh png To dump any errors to a text file from graphicsmagic, run this command:
gm identify 1489.jpg 2> wut.txt Adds a signature or watermark image to an image in an inverse cut-out style (see USAGE).
Run with these parameters:
Example:
addInvertAlphaSig.sh in.png signature_alpha.png Creates a .ppm (plain text file bitmap format) image of a random number of color columns, each column repeating one color a random number of times, to effectively make a vertical stripe of a random width. Colors used can be random or configurable via input file parameter (of a list of hex color values). Generates Z such images. All random ranges, dimensions, and colors to use configurable; see USAGE for script parameters and example.
Run with the following parameters:
Example that will produce minimum 3 vertical stripes, maximum 80, and 5 such images, from the palette sparkleHeartHexColors.hexplt:
randomVerticalColorStripes.sh 3 80 5 sparkleHeartHexColors.hexplt runs SVG2img.sh for every *.svg file in the current directory, additionally passing other parameters which you must specify: see USAGE.
Run with these parameters:
e.g.:
allSVG2img.sh 4200 png 000066 NOTE: to render svgs in subdirectories as well, remove "-maxdepth 1" from the array build code line.
Lists all files that have the tag FINAL in them:
find . -type f -iregex '.FINAL.' or, alternately '.FINAL.'. Paths to files are stripped from the print; you must use a tool like the Everything search engine on Windows to find their folder.
From a path where you wish to list all files that include the offset string FINAL in them, run without any parameters:
findAllFinals.sh If you wish instead to find all files that have the string FINAL (not offest with underscores) in them, run this script with any parameter, for example the word WANGUMB:
findAllFinals.sh WANGUMB Generates a series of images of black and white boxes (noise scaled up with hard edges preserved), then strings them together into a video. Accomplishes this by first generating so many .pbm format (plain-text) images, then converting them (and animating them). Result filename is of format
ffmpeg, mkNumberedLinks.sh
Run this script with these parameters:
Example that will generate images that are 24 columns wide, 16 rows high, and make 1024 such images, and animate them at a source framerate of 5 per second:
makeBWGridRandomNoiseAnim.sh 24 16 1024 5 At this writing, if not always, you must manually specify the target video size hard-coded at the end of this script (in the ffmpeg parameters).
Creates an mp4 video (AVC) from a flat text file list of input image (or video!) file names (one image file name per line in list). Creates the animation at _out.mp4.
This script overwrites _out.mp4 if it already exists.
From the directory with the image animation source images, run this script with these parameters:
Optional: $5 rescale target resolution expressed as nnnnXnnnn. Source images will be rescaled by nearest-neighbor (keep hard edges) option to this target resolution.
NOTES:
The expected list format is, per ffmpeg:
file 0001.png
file 0002.png
file 0003.png
-- etc.
Creates an mp4 video (AVC) from a series of numbered input images. Automatically detects the number of digits in the input frames. Expects only digits in the input filenames. Creates the animation at _out.mp4. NOTE: you may want to use x264anim.sh instead.
This script overwrites _out.mp4 if it already exists.
From the directory with the image animation source images, run this script with these parameters:
Example run:
ffmpegAnim.sh 29.97 29.97 13 png Takes an .svg file and fills all regions of one color (default ffffff, white) with randomly generated colors (not recommended -- random colors can be garish), OR from colors randomly selected from a .hexplt color list (recommended, optional).
Changes (overwrites) input svg file without warning. You may wish to only operate on a copy of the svg file, or make many copies and alter them by calling this script from another script, such as SVGrandomColorReplaceCopies.sh.
Run with these parameters:
Example that will replace every color fill of ffffff (white) in input.svg with randomly generated sRGB colors:
SVGrandomColorReplace.sh input.svg Example that will replace every color fill of ffffff (white) in input.svg with randomly selected colors from eb_favorites_v2.hexplt:
SVGrandomColorReplace.sh input.svg eb_favorites_v2.hexplt Example that will replace every color fill of 000000 (black) in input.svg with randomly selected colors from earth_pigments_dark.hexplt:
SVGrandomColorReplace.sh input.svg earth_pigments_dark.hexplt 000000 SVGrandomColorReplaceCopies.sh), $3 palette file to use (or the word RANDOM). This version of the script adds $4 color to replace.Runs the potrace utility to convert all black and white .bmp images (in the directory in which this script is run) to .svg vector images.
potrace.
Run without any parameter:
BMPs2SVGs.sh Takes an .svg file and fills all regions of one color (default ffffff, white) with randomly generated colors, N times, OR from colors randomly selected from a .hexplt color list. I recommented random color selection from a pleasing palette, because random color picking from sRGB (or possibly any color space) can be garish and not harmonious.
WARNING: this directly modifies (overwrites) the source svg file. (A previous version of this script made a new file name. If you don't want that, copy the svg to a new file name, and modify the copy via this script.
Run with these parameters:
Example that will create 12 randomly colored variations of input.svg:
BWsvgRandomColorFill.sh input.svg 12 Example that will create 12 variations of input.svg with colors randomly selected from RAHfavoriteColorsHex.hexplt:
BWsvgRandomColorFill.sh input.svg 12 RAHfavoriteColorsHex.hexplt: Lists (to a txt file) all files of type $1, by descending order of file size. At this writing, does not list paths--you must use the find command or the Everything search engine utility etc. to locate listed files.
Run this script with one parameter, which is the extension of a type of file you want to list by descending order of size in all subdirectories of the path from which you run this script, for example:
lsTypeBySize.sh png Results are written to a text file named after the file type, e.g. png_files_by_size.txt.
Replaces terminal-unfriendly characters in all files of a given type (parameter 1) in the current directory, via rename.pl. If $1 is not provided, does this to ALL files in the current directory. Why ftun.sh? FTUN stands for "Fix Terminal Unfriendly [folder and file] Names." Terminal-unfriendly characters in file names are any character that may make a script choke if you attempt to pass a file name (or folder name) containing them to a script. See NOTES under USAGE.
Perl, and rename.pl (from http://plasmasturm.org/code/rename/rename) in your PATH, and a Unix or emulated Unix environment.
From a terminal, in a folder with terminal-unfriendly file or folder names, execute with these parameters:
.txt, .png, .hexplt, .ttf, or whatever -- everything found), pass the keyword 'ALL'. NOTE that ALL will cause renaming of directories also.Example that would rename all files with the .png extension to terminal-friendly names:
ftun.sh png Example that would rename all files (regardless of extension):
ftun.sh ALL Example that would operate on all png files in the current directory and all subdirectories:
ftun.sh png SNORFBLARN Example that would operate on all files of every type found in the current directory and all subdirectories:
ftun.sh ALL SNORFBLARN @=\`~!#$%^&()+[{]}; ,-Produces a corrupted version of whatever file you pass to it as a parameter, skipping the first N bytes of the file (see the skipHeaderBytes variable initialization at the start of this script). Designed for e.g. making glitch art from jpg images or mp4 movies. NOTE: because of bash math restraints (unless I use bc, no thanks), this will fail on files of size 100 bytes or less.
Run with two parameters:
Example command that will produce a corrupted version of the file input.jpg, corrupting 60 percent of the data in it:
glitchThisFile.sh input.jpg 60 NOTE: See comment ALTERNATE OPTIONS HERE for other percent options (which are effective for variously sized files)
Runs a script that corrupts all files of a given extension in the directory from which this script is run, producing N glitched e.g. image file variants of all such file types, output to a /_glitched folder. See USAGE for options. Written specifically for the purpose of deliberately making glitch art out of e.g. .jpg files, but it may produce "good" results for a variety of file formats. OR, with some code change, runs BM.exe (Byte Molester, a free tool).
Pass these parameters:
The following command, for example, will select 20 jpg images, make 10 corrupted copies of each, corrupting each copy by 2 percent:
glitchNrandomFilesOfType.sh jpg 20 10 2 Repeatedly calls byte, uh, mangler dot exe to create corrupted copies of all files of a given type from the directory in which it is run. Hard-coded custom parameters at this writing.
DEPENDENCIES bm.exe, uh, byte . . . mangler. The original author named it byte molester, and programmed it to use .fck file extensions. No thanks. And did he want it to be mistakenly thought of as BowelMovement.exe?
Run with these parameters:
Example that loads input.jpg and creating 12 corrupt copies of it:
glitchMangleAllFilesOfTypeNtimes.sh input.jpg 12 Renames all files of type $1 to numbers by reverse sort of file modification date (one use case is this script helping make anims of modifications to files over time).
Run with this parameter:
Example:
renumberFilesByModificationDates.sh png Renames all files of a given extension (via parameter) in the path from which this script is called--renames them to zero-padded numbers matching the number of digit columns of the count of all said files. WARNINGS: 1) use this only in directories where you actually want all files of the given extension renamed by numbers. 2) If any of your file names are numeric-only (e.g. 005.png) before you run this script against them, files may disappear via overwrite, effectively erasing that file by replacing it with new content. For example, a file named 005.png may be overwritten when a file named someOtherFile.png is renamed to 005.png, overwriting the original file named 005.png.
Run with one parameter, which is the file extension you wish for it to operate on, e.g.:
renumberFiles.sh png NOTE: this will choke on file names with console-unfriendly characters e.g. spaces, parenthesis and probably others.
Copies $2 random contiguous bytes from file $1 and writes them to file $3. Could be useful for data bending / glitch art.
Run with these parameters:
Example that reads 15 contiguous bytes from a random location in in.flam3, and writes them to flam3outFragments.bin:
readNrandomBytes.sh in.flam3 15 flam3outFragments.bin dd count=200 bs=1 obs=1 if=/dev/urandom of=random2.bin
For all files in the current directory, attempts to extract basic metadata from exiftool information dumps: File Name (for reference), Exposure Time, F Number, and ISO, and compiles all that to allEssentialImageCameraInfo.txt. Useful for comparing the effects of camera settings on photography.
Run from a directory which has only a given set of photos whose essential camera parameter characteristics you wish to compare. The script will automatically open the resultant tab-delimited text file of data.
Renames all files in the current directory with random character strings of length n (parameter $1, and a default is used if not provided), but with the same file extension.
Run with these parameters:
Example command to rename all files with the extension .hexplt to 20-character random strings:
allRandomFileNames.sh 20 hexplt Creates N ($1) randomly shaped "blob" images.
From a directory you wish to fill with so many random blob images, run with one parameter, which is the number of blobs to make, e.g.:
getRandomBlobs.sh 100 Creates cropped .bmp images from all images of many types in the current directory, such that white area outside black areas is discarded. Useful for preparing art for later conversion to a vector format without wasted border space.
innercrop.sh from Fred's ImageMagick scripts, in your PATH. As those scripts are not freely redistributable, you'll have to download it from the source yourself at: http://www.fmwconcepts.com/ImageMagick/innercrop/index.php
Run from a directory tree full of .png images, without any parameter:
cropAllPNG2BMP.sh Copies the video stream (so, lossless transcoding) of rated electric sheep .avi files (from the electric sheep content folder) into .mp4 videos in a specified directory. Examines the list_member.xml file to do so.
Electric Sheep screensaver, a 'Nixy environment (coded for MSYS2 on Windows)
Set the variables at the start of the script per the locations of various files in your Electric Sheep screensaver install, and run the script without any parameter:
ripAndTagRatedSheep.sh Returns one password of length 44 characters or per parameters you pass to the script.
pass this script two parameters:
Example:
complexPassword.sh 5 44 -- will return 5 passwords of length 44 characters each.
SEE ALSO: http://passwordmaker.sourceforge.net/downloads/cli.html
Tags images with easily customizable metadata; Customizable via a simple editable text file template. Must run prepMediaMetaData.sh and/or other scripts before.
A self-hosted install of polr URL shortener and the API key for it saved to ~/PolrAPIkey.txt, wget, exiftool, Cygwin (this is tied to windows at the moment) . . ?
Run prepMediaMetaData.sh before this script.
This script is NOT DOCUMENTED, sorry. Will document!
Runs utVideoNoSound.sh for every file of type $1 in the current directory. See comments in that script for the effect of this for any one video file.
ffmpeg and a 'nix system (can be Cygwin or MSYS2 for Windows).
Ensure this script is in your PATH, and run it from a directory with avi files that are too huge. Results will appear as UTvideo file names.
Optional parameter $1
utVideoNoSound_all.sh avi If no parameter passed, defaults to avi.
Prepends a random alphanumeric string (but with similar characters excluded) of length $1 to all files in the current directory. As this is a destructive or potentially havoc-inducing action, the script prompts to be sure you want to do this and does not unless you type a given password.
Run with these parameters:
Example that will use the default number of random characters:
prependRandomStringToAllFilenames.sh Example that will prepend random strings 6 characters long to all files in the current directory:
prependRandomStringToAllFilenames.sh 6 Prints a list of unique words in file $1 (parameter), without duplicates, and maintaining order. (the uniq utility only removes consecutive duplicates, but this removes all duplicates). Optionally writes the filtered result over the original file instead of printing it to the screen. To instead sort and deduplicate, use sortuniq.sh.
Run with these parameters:
For example, to print unique words in their original order of appearance from the file gibberwords.txt, run:
getUniqueWords.sh gibberwords.txt To write the result to a file instead of printing it to the screen, pipe it like this:
getUniqueWords.sh gibberwords.txt > gibberwords_deduplicated.txt To overwrite the original file, pass anything as parameter $2, for example:
getUniqueWords.sh gibberwords.txt FLOURBALPER Splits a text file $1, on lines, into smaller files of $2 lines per file, dividing on line breaks. Files may be recombined to form the original file again (see NOTES). Split files will be named after the original and keep the same extension (e.g. .txt or .hexplt), but with numbering in the file names.
Run with these parameters:
Example command that will split a file named aHugoriousTextFile.txt into 4 files:
splitTextFile.sh Humanae.hexplt 256 cat .*Prints $1 randomly generated alphanumeric strings (excluding characters that can be confused for other characters) of length $2. Default values used if no parameters provided.
Run with these parameters:
For example, to print 14 random strings, each 42 characters long, use:
randomString.sh 14 42 To use default settings, omit any or all parameters, e.g.
randomString.sh 8 Or:
randomString.sh Takes an input video, strips the sound, and converts the video stream to an AVI using the lossless utvideo codec.
utVideoNoSound.sh inputVideo.avi utVideoNoSound.sh inputVideo.mp4 etc.
Slices a larger image into X by Y tiles on math of WxH of each slice, via GraphicsMagick convert.
A 'Nixy environment, GraphicsMagick.
Run with these parameters:
Example that cuts input.png to 640x480 slices:
imageGridSlice.sh input.png 640 480 Prepares image metadata text file for insertion into images via ExifTool.
~/metaDataTemplatesPath.txt file, which contains the path to your metadata template files as seen by whatever Unixy tool you use to run this script (e.g. MSYS2 or Cygwin)~_MD_ADDS.txt files which another script will use to set image metadata tags.For all images of many types in the current directory, creates metadata archive files in an _originalMetaData subfolder. Metadata archives are named after the original image.
exiftool, printAllIMGfileNames.sh. Intended archive of metadata to a .7z file will at this writing fail if you're not running Windows (imgMetaDataTo7z.bat).
Hack the script this calls (if you need to, to get a different formats list), then run this without any parameter:
archiveMetadataWin.sh Incremental file number naming by label utility. Finds the highest numbered file having both the phrase FINAL and a five-padded number (nnnnn) in the file name, and renames file names which have FINAL in them but no five-padded numbers; adding incremented five-padded numbers to those file names (to number all such FINAL files by incremented numbers). Handy for incrementally numbering e.g. a lot of new original abstract art work master image file names in a directory tree (operates also on subdirectories), or numbering e.g. abstract works.
Not currently documented. Will document.
Looks for files named after the pattern .*YYYY.[.]MM.[.]DD.[.]HH.[.]MM.[.]SS and updates their modified date stamps to match. Works on all files from the directory tree from which it is run (recursive). Useful for correcting that information if you have for example restored from backups or copied accross drives (which can cause the file timestamps to be made anew, depending on the file system and/or tool), and if you want to examine files by sort of the date they were actually created or modified, not just copied or restored.
This script does not prompt to confirm date stamp updates, it just runs them without asking. Use at your own risk.
Run without any parameter, from a directory tree which you want to so correct the file modified date stamps in:
dateByFileName.sh This script expects terminal-friendly file names. See ftun.sh.
Clears the printed history in the terminal screen (use if your built-in clear command doesn't do the job).
Run withuot any parameter:
clear.sh PERMANENT DESTRUCTION-INDUCING SCRIPT. WIPES THE CONTENTS of all files in the current directory and all subdirectories, making them null or zero byte files. Prompts to enter two passwords before it will proceed.
From a path in which you wish to blank all files to 0 bytes, run this script:
blankAllFiles.sh Converts every .txt file in the current directory and all subdirectories to Unix line endings and utf8 encoding, IF they are not encoded in us-ascii. Overwrites the original files. Needed if some tool or summat creates files in a codepage unusable by tools expecting Unix line endings/utf8.
Run without any parameter:
alltxt2utf8.sh Advanced file renamer.
http://plasmasturm.org/code/rename/rename via http://plasmasturm.org/code/rename/
ftun.sh, preferred, which uses this but with my particular preferences.
To get the path to this script, assuming it is in your PATH, try both of these commands and hope that one works:
which rename.pl
whereis rename.pl Then run this script this way:
/path/to_this/script/rename.pl [options] To preview what would be renamed without actually doing it, use -n:
perl /path/to/this/script/rename.pl -n -z * To see other options and more detailed help, run:
perl /path/to/this/script/rename.pl --help Allows for any AutoHotkey scripts that #Include this script to be dynamically reloaded as executable programs running on the system, for testing-in-development purposes, via keypress of CTRL + ALT + / (forward slash).
Adapted from BETLOG's modification, re:http://www.autohotkey.com/board/topic/122-automatic-reload-of-changed-script/page-2
To cause any script to function as explained under DESCRIPTION, copy this script into the same directory as the script you are developing, and then in your development script, copy the lines of code from this script between the labels " ---- BEGIN AUTO-RELOAD INCLUDE" and " ---- END AUTO-RELOAD INCLUDE," but removing the comments (semicolons) from the start of the code lines in that include section. You may also be able to include this script by not copying it into the same directory of your development script, and only adjusting the include code line to an absolute or relative path to this script. See also the NOTE right after that comment section.
Creates a montage from images selected from a graphical user interface.
Found at and adapted by RAH 2015-06-05 07:41:45 PM from: http://www.autohotkey.com/board/topic/39534-ImageMagick-montage-interface-create-image-tile-sets/
_-Created by: tidbit
_-Enjoy~~!!
ImageMagick's montage.exe must be in your %PATH% for this to work.
As good practice, only ever operate on copies of your images with this script (if something goes wrong, you have your originals still).
Install AutoHotkey, and double-click this script. Follow the prompts/dialogs.
Turns a set of images into 1 big tiled image, or animated images into one big image (depending on limits in either case).
You could also use this as an image converter.
It doesn't matter if you type or don't type the leading period in the FileType field.
Aligns tabs from input data into human-readable aligned columns.
Found at: http://www.autohotkey.com/board/topic/123117-tabalign-tab-align-columnar-data/
Which is an improvement of: http://www.autohotkey.com/board/topic/74885-tabalign-function-to-tab-align-columnar-data/
09/15/2015 05:55:15 PM
I don't know. I never documented it and don't remember ever using this. Note to self: document this if you ever use it again. If it's not useful to you, delete it from this repository.
Force removes all the Windows Store apps that are components (and should not be components) of Windows 8+.
Run from an elevated powershell terminal, from a directory that has this script in it:
interactiveWinAppStoreUninstall.ps You may need to set powershell execution policy first to allow execution of this.
Copy of chocolately install script from chocolatey.org.
Refer to documentation at GitHub and/or chocolotey.org, and/or the file installChocolatey_instructions.txt in this repository.
My custom version of a script that launches MSYS2 (with the settings that I want).
Install in the MSYS2 configuration files, via MSYS2setup.sh, in the same directory as this script.
Takes ownership of a file %1 (parameter 1). See also InstallTakeOwnership.reg.
From a cmd prompt, possibly with Administrator privileges, run with one parameter, which is the name of a file to take ownership of:
TakeOwnFile.cmd stubbornlyInaccessibleFile.txt Takes ownership of a directory %1 (parameter 1). See also InstallTakeOwnership.reg.
From a cmd prompt, possibly with Administrator privileges, run with one parameter, which is the name of a folder to take ownership of:
TakeOwnDir.cmd stubbornlyInaccessibleFolder Wipes the Windows font cache for x32 systems, which can be silly stubborn and render from outdated font files even after you update and reinstall fonts in development, and which font cache service you should have disabled for that reason anyway.
FontReg.exe, from . . . ? I collected this executable from somewhere? It updates Windows font registration or cache or some other magic.
Run via command prompt or double-click:
delFontCache.bat Installs all packages I commonly use from chocolatey.
chocolatey, with all its dependencies.
See installChocolatey_instructions.txt to get chocolatey installed.
Modify the list of packages (broken over newlines in a loop for easier editing) per your wants. (There may also be a commented list of packages to potentially use, which you may want to shuffle into the actual used list.) Then, run this script from a Windows CMD prompt:
installUsedChocolateyPackages.bat To list installed packages, run:
choco list --local-only Enables you to save extremely large files more quickly and continue working on them even while they save. Accomplishes this by monitoring a RAMdisk for changes, and when changes are finished writing to the RAMdisk, syncs (backs up) files from the RAMdisk (nearly instantaneous storage in computer RAM or volatile memory, emulated as a hard drive) to slower storage (a hard drive). The caveat of a RAMdisk is that if you forget to copy files out of it when you're done working with it, they're gone quicker than you can say "No recycle bin." This batch alleviates that problem by continuously copying out of the RAMdrive (to a permanent storage location) in the background. However, even with this there comes a WARNING (see under USAGE).
Because especially huge files (say, gigabytes large) can take much longer to save to a hard drive, even if you use this script (which backs up from RAMdrive to your hard drive continuously), you should take short breaks (perhaps at least a few minutes) every long now and then. Such breaks are good working sense to avoid fatigue anyway, but depending on the speed of your hard drive and the size of files you keep in the RAMdrive, such breaks may be needed to allow your computer to finish writing billions of bytes to disk. I don't know and have not tested what happens if you overwrite the same huge file in a RAMdrive before this batch finishes copying a previously written version of it to a hard drive.
Another way of saying that: if it takes longer than a minute to write the files you work with to your hard drive (permanent storage, not the RAM drive), and you overwrite the same file(s) on the RAM drive more than once a minute (by saving it again), which might mean you save to the RAM drive before this batch finishes copying it out of the RAM drive to the permanent staging location (SYNC_TO_DIR), this batch may never finish writing any file to permanent staging for far longer than you hope. Maybe not until you give the RAM drive enough rest time for it to do so. I don't know. Hopefully I will update this warning if I test that.
Do in fact read that WARNING. To set up the environment to use this script and actually use it:
Obtain, install and configure a RAM disk (such as from the URL given under DEPENDENCIES), as large you need and is suitable for your system.
Create a folder which you will want to regularly and automatically sync files out of the RAM disk and into (continuous backup). For example: C:\Users%username%\Desktop_RAMdrive_mirror (where %username% is a Windows environment variable that means "your user folder." If you type %username% into Windows explorer and press
Set the source (RAM drive) and destination (permanent hard drive storage) folders for this batch file in the first lines of code of RAMdriveLiveSync.bat, to folders which actually exist via your RAM drive and a hard drive, for example (and what it is hard-coded to by default) :
SET SYNC_FROM_DIR=V:\ SET SYNC_TO_DIR=C:\Users\%username%\Desktop\_RAMdrive_mirror
Where SYNC_FROM_DIR will be the RAM drive (source) and SYNC_TO_DIR will be a permanent storage staging area (which should be a folder on a hard drive, not a RAM drive).
With all that done, you're ready to work with files actively in the RAM drive source folder (a virtual folder created in memory!), after you double-click or otherwise run this batch:
RAMdriveLiveSync.bat
Then, leave this batch running for as long as you work with the files in the RAM drive. (This means leave the command prompt open -- or the terminal or "DOS box" or whatever you want to call it, running this batch--leave it open. Once per minute, the batch will copy all of their changes out of the RAM drive into the permanent staging location (SYNC_TO_DIR).
When you are done working with the files, and they have synced to SYNC_TO_DIR, you will probably want to further relocate them to other permanent hard drive locations that make more sense.
Previously I recommended the freeware program RAMdisk, available from http://www.dataram.com/ but that software was nerfed. The version I obtained allowed the RAMdisk to be up to 4GB in size, but the newest version now only allows up to 1GB. The free (and vs. the RAMdisk software I tested, easier to use) software I link to an obtain domain under DEPENDENCIES has no such limitations.
Installs pyenv on Windows. NOTE: I have not gotten good use of pyenv on windows yet. I reinstalled in several different ways and finally got it to change python versions, but then pip didn't work. re: https://github.com/pyenv-win/pyenv-win
windows, git, setenv, and modpath (the latter two from _ebPathMan or elsewhere).
Run from an administrator command prompt:
pyenvInstallWin AFTERWARD, restart a command prompt with administrator privileges, and run:
pyenv install x.x.x --where x.x.x is the python version you want. Then run:
pyenv global x.x.x (With the same version again for x.x.x.) Then run:
pyenv rehash You may need to run that rehash command every time you change the global version.
Prints the enabled network adapter speed.
Run without any parameters:
get_windows_network_enabled_adapter_speed.bat Tests the performance of a disk associated with a Windows install.
RUN WITH one parameter, which is the drive letter (no : or \ in it) you want to test, for example:
winsat_drive_test.bat C Periodically backs up a source directory structure (with files) into a .7z archive, named by an archive target name and the the date and time of the backup. The source folder to back up, the target archive name, the backup interval, and an optional process to suspend are all easily user-configurable--see the "User-specified variables" comment below.
These need to be in your PATH:
Hack the SOURCE_DIR and DEST_DIR variables (in the first SET commands after the CODE comment) to the appropriate directory you wish to archive. This batch will create the archive directory if it does not exist. Also optionally set the name of an executable which manipulates the files you wish to archive (for example, set the command to SET PAUSE_AND_RESUME_EXE=Minecraft.exe), so that process.exe will pause that process before backup and resume the process after. (This is to avoid the problem of backing up files which fall out of sync with other files during backup--if in fact that's a problem. Hence the "optional.") Also, set BACKUP_MINUTES_INTERVAL to the number of minutes you wish for this batch to wait between backups.
With those preparations in place, run this script without any parameters:
MonitorAndBackupDirTo7z.bat
Finally, you can cancel execution of this batch from the Windows console by pressing CTRL+C, then Y.
Super toasts the security problem which RDP is, by deleting ALL files on the system drive which have "Mstsc" in the file name, AND deleting associated services.
From a command prompt with administrator privileges, run this script:
toastWindowsRDP.bat Runs the tronscript for Windows debloat / disinfect / maintenance super-script, with preferred switches.
tronscript (you'll find it with an internet search)
Copy this script to the same folder as the extracted tron.bat. Run from a cmd prompt (maybe as an Administrator):
run_tronscript_debloat.bat Deletes an over-the-top so wrong nag to update Windows 7 to 10 (which was an absolute fiasco for many users), which Microsoft pushed on all users via an update.
Run this batch without any parameters:
delwin10nag.bat NT Super User, or NTSU. Opens a CMD or Windows "command prompt" with "NT Authority/System" privileges. It may allow you to do things an Administrator terminal can't.
paexec may need a particular Windows service to be running as it uses "remote" execution locally.
You may need to run this from a command prompt or account with administrative privileges, or right-click it and "Run as Administrator:"
NTSU.bat Performs a Windows image repair / cleanup, then system file scan and repair.
From a command prompt with administrative privileges, run this script:
windows_DISM_onlineRepair.bat I don't know what this does. I also don't care anymore. This script is vestigial. MSoft toasted gadgets at some point; this script or "batch file" can only apply to older versions of Windows.
Run without any parameter:
ResetWin7Gadgets.bat Created by: Shawn Brink
http://www.sevenforums.com
Tutorial:
http://www.sevenforums.com/tutorials/140358-gadgets-not-displaying-correctly-windows-7-fix.html
Runs ahkrip.bat (SEE) for all .ahk files in the directory from which this script is called.
You must have ahkrip.bat in the same directory as this (or possibly just in your PATH).
From a directory with one or more .ahk (AutoHotkey) scripts ready to be compiled to executables, run this script:
call-ahkrip.bat See USAGE in ahkrip.bat for expected results.
Compiles executable %1 (parameter 1), which should be an .ahk (AutoHotkey) script, to an executable, via Ahk2exe.
Ahk2exe installed somewhere and in your PATH.
This will erase and recreate any ahk-generated .exe files in the directory in which it is run!
Run with one parameter, which is the file name of an AutoHotkey script ready to compile to an .exe, for example:
ahkrip.bat tweak_mouse_keepalive.ahk Resultant executables will have the same base file name as the script compiled.
Updates or creates all_originalMetaData.7z with metadata .xmp sidecars from _originalMetaData folders.
Before you run this script, run archiveMetadata.sh. Then run this without any parameter, at any time you want to update the metadata archive:
imgMetaDataTo7z archiveMetadata.sh..xmp sidecars in folders with _originalMetaData in their name (and their subfolders) to an all_originalMetaData.7z archive, without overwriting .xmp sidecars that already exist in the archive (or, preserving already existing .xmp sidecars). In the case of .xmp sidecars which are in a path, but which are not in the .7z archive, they will be added to the .7z archive..xmp sidecar in the archive, you would then first manually delete it from the archive; the newer/modified version of the sidecar on disk (in a path) will then be re-added to the .7z archive. What are .xmp sidecars? They archive metadata from associated image (and video!) files.Backs up a Corel Graphics Suite FontNav font group database from hard-coded directories?
WinRar? I would like to update this to use 7z.
I don't know. Rework this if you use it at all. See list under CODE comment
From a command prompt, run this script, or just double-click it from Windows Explorer:
Backup_Font_Group_Database.bat
Escapes troublesome characters in an arbitrary text file (parameter %1%), in-place, so that the text can be used in a batch script without errors.
Run this batch with one parameter, which is the file you want to
have escape sequences added to in-place; e.g.:
escapeTextFileString.bat wonderousEvolutionOfPants.txt Overwrites file %1% without warning. Backing it up before running this
batch is advised.
A 'Nixy environment (e.g. Cygwin or MSYS2) on Windows.
Terminates a wide variety of processes which may be found on Windows computers which I often consider extraneous or even resource hogging, including stopping such services. I don't recommend anyone to just run this script nilly-willy unless you are very sure you don't need the processes in it to run (as I am sure for myself, but I have only examined them for my own purposes, which my vary from yours).
process.exe Command Line Process Viewer/Killer/Suspender for Windows, from beyondlogic.org installed in your PATH.
Run without any parameters:
killHogApps.bat Force terminates all running cmd (Windows command prompt) processes.
Run without any parameters:
killAllCMD.bat Attempts to repair causes of Windows Update hanging / never finding / installing needed updates.
Run from a command prompt with administrative privileges, without any parameters:
fixWindowsUpdateHang.bat READ THE WARNING under usage. This batch script cleans unnecessary junk files--often anywhere between ~1-16 GB or more!--from a typical fully updated and well-used Windows installation. It does this by way of the built-in ROBOCOPY command, instructed to sync temp folders with an empty folder in multiple threads (faster than rd/rmdir). You may wish to read all of the REM comments in this file before running it. By junk I mean files which will not ever be used; e.g. *.dmp *.temp, *.tmp, and optionally all *.bak files on the drive. (Side note: NTLite will get you still ~4GB more garbage cleanup!)--but use that cautiously.)
MIND THE WARNINGS. If you use this:
Run this as an Administrator, or from a console with Administrator rights:
RoboWipe.bat Autohints all ttf format files in the current directory, via ttfautohint.exe.
If it is important to keep earlier revisions of files that this script would update, only run this script on files that you have backed up. It overwrites files without warning!
From a command prompt opened to a directory with TrueType font files to be autohinted, run this script:
ttfAutoHintAll.bat Uninstalls "Command Prompt Here" right-click menu for folder.
Double-click this .reg file and click "Yes" to merge it into the registry, then click "OK."
Installs "Command Prompt Here" right-click menu for folder.
Double-click this .reg file and click "Yes" to merge it into the registry, then click "OK."
Disables nags to be certain you want to open files originating from a network source which you trust but which Windows doesn't.
Created by: Shawn Brink
http://www.sevenforums.com
Tutorial: http://www.sevenforums.com/tutorials/182353-open-file-security-warning-enable-disable.html
Import this reg file by right-clicking it and clicking "Merge," or by double-clicking on it.
Uninstalls right-click menu options on folders and files to open an MSYS2 terminal (normal install or alongside Ruby 2.7) in that location.
Right-click this file and click "Merge," or double-click it, then click "Yes" and "Ok."
Installs right-click menu options on folders and files to open an MSYS2 (as installed alongside Ruby 2.7) terminal in that location.
Right-click this file and click "Merge," or double-click it, then click "Yes" and "Ok."
Installs right-click menu options on folders and files to open an MSYS2 terminal in that location.
Right-click this file and click "Merge," or double-click it, then click "Yes" and "Ok."
Installs a right-click menu in Windows Explorer to take ownership of any folder and all its contents. Helps when permissions go wonky or anything locks you out of seeing or modifying a folder.
Import this reg file by right-clicking it and clicking "Merge," or by double-clicking on it.
Many of these scripts have dependencies beyond what are included in your typical GNU core utilities.
To run any of them your operating system must be capable of that and/or you must be able to install things on your operating system to run them, for example:
Python for .py scriptsbash for .sh scriptsDOS for .bat filesMany of these scripts were tested and found to work with both a Unix emulation layer on Windows (such as Cygwin or MSYS2), and also MacOS. Very few of the scripts have been tested on any Linux or other GNU variants, but they will probably work on many of those.
All the scripts are designed under the assumption that they can be located by your operating system via the PATH environment variable. If you're not sure what that means, see these tutorials:
The scripts can be either dynamically added to the PATH on terminal launch via terminal customization, or permanently (for example as SYSTEM variables in Windows, or via bash profile customization on Unix variants). Sometimes a script calls another script or utilities assumed to be in your PATH.
Many of the bash (.sh) scripts that don't call other scripts may be usable locally (if not in your PATH) if you cd into their directory, or copy them to a directory with files they would operate on, and then run them with ./ and then the script file name immediately after that, and then any parameters they require as documented in each scripts' USAGE section. ./ means "this directory" to the terminal. For example, if randomString.sh is not in your path, you may copy it to any directory, open that directory in the terminal, and run it with ./ before it, like this, to create 12 random strings of length 34:
./randomString.sh 12 34
GG67Apt2GnpDkyDPAGn3tPmeAsjACazrRe
4UYs7tWJWzdueZwvhgpp3hnRzHpGYJQSNt
7AtkAWXMNQb2pqQY5UgZeAAfWGEwSemsjs
ZwVSdGpvdmyckfVFMWVgWvxHWVsSmygb5N
uGh3hpVdsrJ8QqNjFueem8kMyBF6DvhGnm
AkMgBH68NUAmtQtQ3Jb5RzJuDAW28BqnEq
d2Tw9jhSRjZDeRpGxfADH5EGDsd9mA6skP
wmehvT7aq4ZuFRyhYdvYQgZb4yRCjMagES
MKTna3uFJVx7KbbPD7jRhMEWxBC8nfyRGA
rhaMnY5AUKxNdtMJEEPvNKEmgW3hq9Tn6Z
UH64yNkMHs2uPvMtr3D8MfY2XSc39zdBrr
E9XZyEZpzb4WhFdmTMWpeSTcSxgcBpDk9fYou may want to use utility scripts I wrote and/or utility binaries to get these scripts into your PATH. Explanations of those follow.
See the README.md for \_ebPathMan for instructions on how to use scripts/utilities in that repository to get collections of scripts etc. into your PATH, either permanently or dynamically.
A collection of binary utilities etc. See that repository's README.md.
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
macDevSetup.sh and installUsedBrewPackages.sh. Many scripts in this repository are designed to use versions of utilities that those scripts will get into your PATH, and will not work without that setup.Python: I manage python versions on Mac via pyenv (which I prefer over asdf, which I think I used and ran into trouble)nodejs: I manage node versions with asdf.Examine the scripts etc. in /scripts/admin.
This repository has scripts that make use of custom file formats. Ones that I remember as I write this (there may be others) are:
.hexplt.rgbplt.cgpA .hexplt file is a list of RGB colors expressed as hexadecimal color codes, preceded by a pound or hash sign, one per line. You will find a large collection of these in my _ebPalettes repository. I acknowledge that many of them have horrible file names that nobody would be able to remember, but the best I can do for so many random palettes is to randomly name their files. Here are links to:
This format may already exist under another name somewhere else; I don't know. I am aware that there are other palette formats, but I found them opaque (pun not intended), so I contrived this format. Other formats I have encountered are binary (machine-readable only). I don't see that as useful for easy reference and use by scripts.
An .rgbplt format file is analogous to .hexplt but without any # at the start of list items, and it lists decimal RGB values: three numbers between 0 to 255 per line, separated by spaces.
A .cgp file is a color growth preset to be used with color_growth.py. Examine the usage comments in the source code of that script and/or the output from python color_growth.py --help for details.
For some idea of how arguments to scripts work, examine this tutorial and this StackOverflow answer. Arguments to bash scripts work the same way as arguments to bash functions.
Two possibilities for parameters to scripts are positional parameters and named parameters.
Positional parameters must always specificy exactly the right kind of argument in exactly the right order, like this example, where the structure of the command to run the script is extremely strict, like scriptName numericArgument stringArgument:
script.sh 14 SNORFBLURNNamed parameters may be designated with switches, and offers more flexibility, with a run command structure like scriptName [-n numericArgument] [-s stringArgument]:
script.sh -s SNORFBLURN -n 14(Note how the order of -n and -s were switched there: named arguments may be in any order).
The vast majority of scripts in this repository take positional arguments (or parameters), for these reasons:
On one hand, I only want to write things in this section that I would have difficulty remembering (I am motivated to write things down that I would forget; if it is knowledge I take for granted, I don't have a need to write it down for myself). On the other hand, I'm interested in providing information that might be educational to others who are learning. The former purpose may usually win.
Also, I could be quite wrong about the best or proper way to do anything, and I can't guarantee the suitability of this information for any purpose.
A history of a kludge: I have gone back and forth on using versions of four ported tools from Unix: sed, find, sort, and uniq as copies from various windows/Mac ports but renamed as gsed, gfind, gsort, and guniq, in a subfolder of the \_ebSuperBin repository which I include in my PATH on Windows. But I found that MSYS2 updates would leave me with errors about possible cygwin1.dll version conflicts, and broken script runs. So I stopped copying/renaming those GNU utilities to that repo, and instead I use the ones as provided by MSYS2, which it keeps current and in the MSYS2 user bin folder(s), and as originally named (I don't rename them). But there may be scripts in /_deprecated that have the names I don't want to use anymore; to further develop or revive any of those use the proper (not g-prefixed) names.
Note that tools and scripts in this repository are subject to high flux, because I may edit and develop them as I use them or discover bugs, and/or because I may freely add or remove anything from this archive. Generally I move anything not useful (or redundant) to the \_deprecated folder. Things under development or suspended for bugs are in the \_in_development. folder. I have even deliberately rewritten git history at times in the project (to cut down repo bloat).
For easier interoperation on various platforms, most of the files in this repo may have Unix line endings, even if they are developed in Windows. I configure my text editor to create Unix line endings by default (which many free and advanced text editors can do). This is only important if you open a file in a text editor that doesn't even know what Unix line endings are. Any modern editor worth using will transparently adapt to whatever line ending style is in any file.
If you want to create computer scripts that can be run on multiple operating systems, some portability concerns (the ability to run a program on different operating systems, or the ability to easily port it to do so) come into play. So do opinions about scripting languages and the utility of them even if they are portable.
When I discovered I can emulate bash scripting via Cygwin on Windows, and that the scripts had hope of running on Unix variants (like MacOS and Linux), and that the bash scripting language is so much more elegant and easy to use, I forsook .bat scripts and only looked back when necessary.
Python and Processing are also cross-platform.
If anyone has brought Windows' DOS/CMD/.bat scripting to other platforms, I don't know why and I frankly might not care.
Windows encodes line endings in text files etc. differently than Unix and Unix variants. Some Unix tools ported to Windows return and manipulate Windows line endings, where other Unix tools don't. The result is that line endings are incompatible between tools that look for one type vs. tools that look for another type, and it breaks things. I have had to dig to that as a root cause of data processing errors many times, but switching to MSYS2 (vs. Cygwin) greatly reduced those headaches.
The tr tool can delete Windows newlines from a stream like this:
cat fileWithWindowsNewlines.txt | tr -d '\15\32'dos2unix will also change Windows newlines (and possibly other Windows-specific problems like unexpected codepages):
dos2unix fileWithWindowsNewlines.txtMSYS2 is a lightweight GNU emulation toolset and compiler environment for Windows. So is Cygwin. But Cygwin is less lightweight. In early development I used only DOS batch scripts. Later I discovered Unix emulation via Cygwin, and oh how liberating and how much easier bash scripting was. But Cygwin gave me nightmares related to Windows newlines, and I looked for something better and found that in MSYS2.
See install_MSYS2_right_click_menu.reg and/or my fork of msys2-mingw-shortcut-menus.
To clone an open MSYS2 terminal in the same path (for example without interrupting a long run of a script), and launch Windows' file explorer in that path:
Right-click MSYS2 terminal title bar
Click "New"
In the open terminal, type:
start pwd
This uses the fact that the Windows shell environment will interpret a command which is a path to simply open that path in file explorer, and that bash takes something surrounded by backticks to mean "execute this, then pass it to the previous command. pwd returns the current path, and MSYS2's terminal automagically converts the unixy path to a Windows path, and passes it to the command start.
For switches and named parameters, see how things are handled in color_growth.py.
The below example is copied from paletteCompareCIECAM02.py. It checks the length of sys.argv to identify whether a required parameter was provided, and if not, prints a helpful error and quits with an error code.
import sys
if len(sys.argv) > 1: # positional parameter 1
hexpltFileNameOnePassedToScript = sys.argv[1]
else:
print('\nNo parameter 1 (source .hexplt palette file one) passed to script. Exit.')
sys.exit(1)
if len(sys.argv) > 2: # positional parameter 2
hexpltFileNameTwoPassedToScript = sys.argv[2]
else:
print('\nNo parameter 2 (source .hexplt palette file two) passed to script. Exit.')
sys.exit(1)Piping with the | operator is very common in scripting operations that I do. A pipe passes the result of the command on the left to the command on the right. A lot of Unix and Unix-like tools are designed to do this. Here is documentation on pipe or pipeline operators.
I also use the redireciton operators < and >. Here's a link to documentation on the redirect operators.
Better than the pipe operator in many cases is the here-string operator <<< which passes the result of the command on the right to the command on the left. Here is a link to documentation on that (along with everything else Bash.)
Bash has a built-in variable, $?, which is assigned the error level (or exit code) of the previous command. By convention when there was no error with a command, error level is set to 0.
So, for example, to check whether an executable was run successfully (and thereby infer that it is in the PATH and probably works as expected), you can call any executable and then check the error level variable:
ffmpeg --help
if [ "$?" == "0" ]; then echo "No error (error level 0) after run of ffmpeg. Assumed to be in PATH and working."; fiIf I want to check that a command succeeds, but I don't want to print the output of the command, I'll redirect both stdout and stderr to /dev/null:
ffmpeg --help &>/dev/nullOver here is a tutorial on stderr/stdout redirection.
Or to redirect both stderr and stdout to a file:
some command &>log.txtSee getFullPathToFile.sh to work around the problem of which and whereis being unhelpful finding paths to things on different platforms.
In bash, you can create variables and initialize them with values returned from commands, via command substitution.
A practical example is to get a random number in a range, and assign it to a variable. By piping commands and results from seq, shuf, and head -n 1, we can obtain a random number in a range. seq will print numbers in a sequence, like this:
seq 1 5Example result output:
1
2
3
4
5If we pipe that seq command to shuf (with the pipe operator (|), it will take the lines of the input stream and rearrange them randomly, then print them:
seq 1 5 | shuf
4
3
2
5
1If we pipe that to head -n 1, it will print only the first line of the output:
seq 1 5 | shuf | head -n 1
3(If you piped it to head -n 2, it would print the first two lines, and head -n 3 would print the first three, and so on).
If we enclose the series of commands in a dollar sign and parenthesis (which is called command substitution), it is evaluated and we can assign the result to a variable, like this:
randomNumber=$(seq 1 5 | shuf | head -n 1)
echo $randomNumber
4You can also evaluate and assign it to a variable via backticks, like this:
randomNumber=`seq 1 5 | shuf | head -n 1`But you may run into problems, depending, if you use backticks for command substitution. Here are sources on that: (1) (2)
A perhaps broader tutorial and reference is over here.
To create an array from every file of a type, e.g. png, use the find command (here, including a print command that chops off the leading ./ from every result:
array=( $(find . -maxdepth 1 -type f -iname \*.png -printf '%f\n') )The second set of parenthesis is explained in this this StackOverflow answer.
Previously I have created variables by enclosing a command in backticks, but I've learned that can present undue difficulties. Also, I've previously not enclosed the whole command substitution (of the syntax $() in further parenthesis (like ( $() ). To not enclose it in parenthesis, I think, leaves the result as a string (collection of characters), which you may be lucky (depending on how you create it, and maybe also depending on the "Internal Field Separator" or IFS) to be able to count or iterate over. Enclosing it in parenthesis makes it an array.
Another way to create such an array, but sort by file date (and without explanation of sort and sed here):
array=( $(find . -name "*.png" -print0 -printf "%T@ %Tc %p\n" | sort -n | sed 's/.*[AM|PM] \.\/\(.*\)/\1/g') )A way that may work better where the result list would be extremely long (which can cause "too long" errors, depending) is to redirect the result via > to a file, then scan every line of the file and perform an operation related to that $line, like this:
find . -maxdepth 1 -type f -iname \*.png -printf '%f\n' > allPNGs.txt
while IFS= read -r line || [ -n "$line" ]; do
echo "$line"
done < allPNGs.txtAlso, this can work:
readarray -d '' array < <(find . -name "*.png" -print0)Which results in an array named array. Or this, to create an array from every line of a text file:
array=( $(<inputFile.txt) )Again note the extra parenthesis. They become important if you want to know the length of the array, which is 1 if you don't use parenthesis, because it is not an array if you don't use parenthesis. With parenthesis, the length may be more than one.
Arrays created by all of the above methods are subscriptable; you may access the array elements by index; here is an example for index 4:
echo "${array[4]}"
SNEERFBLURN.pngTo iterate over elements of an array and do something with each element:
for element in ${array[@]}
do
echo $element
done
chalf
flibflub
chulfor
hooligan
plibplupTo get the length of an array use # inserted in the expression that means "all elements of the array," as follows. These commands create an array of all file names that end with .txt in the current directory via a find command, then echo the length of the array:
arr=(`find . -maxdepth 1 -iname \*.txt -printf '%f\n'`)
echo ${#arr[@]}Example output:
8Suppose you have an array named directories, which is the names of all (sub-)directories in your working folder, which you can verify by looping over its contents:
for element in ${directories[@]}; do echo $element; done
.
flarf
fleur
florf
flurfSuppose also that if you loop over the array and try to do something with the directory . (the first element in the array, which is the current directory), your script would do something unexpected, so you don't want that first element of the array. You want to remove it. You can do that like this:
directories=(${directories[@]:1})This might be called "array slicing," I don't know. :2 will list everything but the first two elements of the array, :3 will list everything but the first 3, and so on.
I have not had occasion, using bash, to remove or insert specific elements into an array at specific indices. Python does that more easily.
This may be kludgy but it reads the most elegantly in my opinion, from here -- and more importantly, in my environment at least, it works:
reversedArray=($(echo "${allIMGsArray[@]} " | tac -s ' '))Adapted from TheGeekStuff; if there are duplicates between the arrays they will appear more than once in the new array:
FLORFELF=('BLORB' 'CHURF' 'LORGL' 'HORCHUF' 'BLEURG');
BLUBARG=('CHOWF' 'HULP' 'GLOR' 'GLARG' 'FLORG' 'MURG' 'BELG');
OMNIBLARG=("${FLORFELF[@]}" "${BLUBARG[@]}")To count the number of files of a given type, for example png, run this command:
ls *.png | wc -lcount.sh is a shortcut to this; pass it only the file type with no period:
count.sh pngcount.sh prints the result:
42If you have a variable named filename with a value 2020_07_15__08_32_50____62b144_colorGrowthPy.png, this will create a new variable named fileNameNoExt without the .png at the end (just 2020_07_15__08_32_50____62b144_colorGrowthPy:
fileNameNoExt=${filename%.*}If you have a file named image.png stored in a variable named filename, this will get the file extension (png and store it in the variable fileExt:
fileExt=${filename##*.}This takes a variable whose (string) value is a path to a file and then a file name, and isolates it to just the file name (no path), and assigns it to the variable fileNameNoPath:
fileNameWithPath='2djio/fefe/fjifeif.txt'
fileNameNoPath="${fileNameWithPath##*/}"
echo $fileNameNoPath
fjifeif.txtThis can also be done with basename and command substitution (fileNameNoPath=$(basename $fileNameWithPath)), but I prefer this as it avoids even calling an executable (the echo command here calls an executable, but that is only to demonstate that it worked. In a script it could be a command with or without echo that makes use of the variable).
This takes a variable whose (string) value is a path to a file and then a file name, and isolates it to just the path to the file (no file name), and assigns it to the variable pathNoFileName:
fileNameWithPath='2djio/fefe/fjifeif.txt'
pathNoFileName="${fileNameWithPath%\/*}"
echo $pathNoFileName
2djio/fefeThis can also be done with dirname and command substitution (similarly to the $fileNameNoPath example), but here again this avoids calling an external executable (it may be faster).
This will store the full path of the current directory in a variable named fullPath:
fullPath=$(pwd)This will get the name of the current directory, without the path to it, and store it in a varaible named currentDirNoPath:
currentDirNoPath=$(basename $(pwd))(That was nested command substitution.)
Sed is a GNU (Gnu is Not Unix) core utility. General sed notes:
The sed (stream editor) tool can work with input from a statement made before it via a pipe (|):
An example using the echo tool, which prints to the terminal any statement after it:
echo fleur fleur chalp fleur chulp | sed 's/fleur/flarf/g'That command results in every instance of the word fleur being replaced with flarf:
flarf flarf chalp flarf chulpThe s in that command means search, and the g at the end of it means "replace every instance of the replacement expression found." Everything between the first and second slashes (/) is the search expression, and everything between the second and third slashes is the replace expression.
Some versions of sed work okay without surrounding the command/search/replace expression with single or double quotes, but some don't, so I always surround them.
To overwrite a file with the results of a sed stream edit, use the -i switch with sed, and give the file name after the sed search/replace expression:
sed -i 's/fleur/flarf/g' wut.txtIf you omit the -i flag, sed leave the file as-is and print the stream edit result to stdout.
I often use regex groups for sed, like:
(.*)That means "group any character (.), any number of times (*)).
I also use character count ranges, like:
[a]{2,}That means "two or more a characters) in sed. But in the terminal and scripts, because parenthesis (()) and curly braces ({}) have programmatic functions, they must be escaped for sed to use them instead of the terminal. Those are escaped by prefixing them with a backslash (\).
So, the previous two example expressions, escaped for sed, become:
\(.*\)
[a]{2,}To make a character or group in an expression optional (so that it may be matched zero or more times), use {0,} after a character or group in the match expression:
[a]{0,}That means "zero or more of the character a".
Practical example: delete comment markup (#) from the start of line, and optionally also delete any space after the comment marker (# ), but keep the rest of the line (the rest of the line captured with (.*) and put into the replace expression with \1:
echo '#flurfefl' | sed 's/^#[ ]{0,}\(.*\)/\1/g'These are notes on a very particular text processing case I had for my wants for this documentation, but it references things that might be useful in other cases.
Before arriving at this I ventured off into Perl instead to try to solve this; I had no success. The match I wanted was easy-peasy in regex101.com, but the Perl CLI did not agree. Code golf: anyone able to do this with Perl?
My criteria:
Reference:
The sed expression:
sed -i -e ':a;N;$!ba;s/\( {2,}[^\n]*\n\){2,}/\n&\n/g' parse.txtWhat that expression does is break any lines with at least two spaces at the start of them, and which are one after another:
Blah blablabla blah mcbla blah blah
img_01.ppm
img_01.png
img_02.ppm
img_02.png
Blor mcblorblah blahblah blah mcblor
more blor mcblorblah
img_01.ppm
img_02.ppm
img_02.png
img_03.ppm
More blahdy blah blah blahinto a group with newlines before and after the group, but no newlines within the group:
Blah blablabla blah mcbla blah blah
img_01.ppm
img_01.png
img_02.ppm
img_02.png
Blor mcblorblah blahblah blah mcblor
more blor mcblorblah
img_01.ppm
img_02.ppm
img_02.png
img_03.ppm
More blahdy blah blah blahThe CMD prompt and Windows do not store a permanent command history, which is a feature I find very handy. However, a third-party tool can provide it! If you use the CMD prompt, I strongly suggest you install clink (I install it via chocolatey), which provides this feature. With clink installed, you can close a CMD prompt, open a new one, type the up arrow, and see your previously typed command (and ones before it with further up arrow presses, and cycle to newer ones again with down arrow key presses). Without it (as Windows/CMD is natively built), you don't get that.
From an MSYS2 terminal, type and enter:
`cmd`The Windows command prompt starts right in the MSYS2 terminal interactively.
To install a right-click menu for folders and folder backgrounds to open that folder in Windows CMD, double-click this registry import file, then click "Yes" and "OK:"
installWindowsCMDhere.regAnd/or:
ALT+D to quickly jump there.cmd, then press A Windows cmd prompt will open in that directory.
And/or see StExBar.
All these tips re an "off-topic" post at StackExchange.