Module CNLib.cnlib.cnfunctions

A collection of common functions used by CN software

Functions

pascal_case: Convert a class name to it's Pascal equivalent do_bool: Convert other values, like integers or strings, to bools dpretty: Pretty print a dict lpretty: Pretty print a list pp: Pretty print a dictionary or list combine_dicts: Update a dictionary with one or more dictionaries sh: Run a command string in the shell load_dicts: Combines dictionaries from all found paths save_dict: Save a dictionary to all paths

Functions

def combine_dicts(dicts_new, dict_old=None)

Expand source code

def combine_dicts(dicts_new, dict_old=None):
    """
    Update a dictionary with entries from another dict

    Args:
        dicts_new: A dictionary or list of dictionaries containing new
        keys/values to be updated in the old dictionary
        dict_old: The dictionary defined as the one to receive updates

    Returns:
        The updated dict_old, filled with updates from dict_new

    This function takes key/value pairs from each of the new dicts and
    adds/overwrites these keys and values in dict_old, preserving any values
    that are blank or None in dict_new. It is also recursive, so a dict or list
    as a value will be handled correctly.
    """

    # default return val
    if dict_old is None:
        dict_old = {}

    # sanity check
    if isinstance(dicts_new, dict):
        dicts_new = [dicts_new]
    if len(dicts_new) == 0:
        return dict_old
    if not dict_old:
        dict_old = {}

    # go through the new dicts in order
    for dict_new in dicts_new:

        # for each k,v pair in dict_n
        for k, v in dict_new.items():
            # copy whole value if key is missing
            if not k in dict_old:
                dict_old[k] = v

            # if the key is present in both
            else:
                # if the value is a dict
                if isinstance(v, dict):
                    # start recursing
                    # recurse using the current key and value
                    dict_old[k] = combine_dicts([v], dict_old[k])

                # if the value is a list
                elif isinstance(v, list):
                    list_old = dict_old[k]
                    for list_item in v:
                        list_old.append(list_item)

                # if the value is not a dict or a list
                else:
                    # just copy value from one dict to the other
                    dict_old[k] = v

    # return the updated dict_old
    return dict_old

Update a dictionary with entries from another dict

Args

dicts_new

A dictionary or list of dictionaries containing new

keys/values to be updated in the old dictionary

dict_old

The dictionary defined as the one to receive updates

Returns

The updated dict_old, filled with updates from dict_new This function takes key/value pairs from each of the new dicts and adds/overwrites these keys and values in dict_old, preserving any values that are blank or None in dict_new. It is also recursive, so a dict or list as a value will be handled correctly.

def compare_versions(ver_old, ver_new)

Expand source code

def compare_versions(ver_old, ver_new):
    """
    Compare two version strings for relativity

    Args:
        ver_old: Old version string
        ver_new: New version string

    Returns:
        An integer representing the relativity of the two version strings.
        S_VER_SAME means the two versions are equal,
        S_VER_NEWER means new_ver is newer than old_ver (or there is no old_ver), and
        S_VER_OLDER means new_ver is older than old_ver.

    This method compares two version strings and determines which is older,
    which is newer, or if they are equal. Note that this method converts
    only the first three parts of a semantic version string
    (https://semver.org/).
    """

    # test for new install (don't try to regex)
    if not ver_old or ver_old == "":
        return S_VER_NEWER

    # test for equal (just save some cpu cycles)
    if ver_old == ver_new:
        return S_VER_SAME

    # compare version string parts (only x.x.x)
    res_old = re.search(R_VERSION, ver_old)
    res_new = re.search(R_VERSION, ver_new)

    # if both version strings are valid
    if res_old and res_new:

        # make a list of groups to check
        lst_groups = [
            R_VERSION_GROUP_MAJ,
            R_VERSION_GROUP_MIN,
            R_VERSION_GROUP_REV,
        ]

        # for each part as int
        for group in lst_groups:
            old_val = int(res_old.group(group))
            new_val = int(res_new.group(group))

            # slide out at the first difference
            if old_val < new_val:
                return S_VER_NEWER
            elif old_val > new_val:
                return S_VER_OLDER
            # parts are equal, go to the next one
            else:
                continue
    else:
        raise OSError(S_ERR_VERSION)

    # return same if all parts equal
    return S_VER_SAME

Compare two version strings for relativity

Args

ver_old

Old version string

ver_new

New version string

Returns

An integer representing the relativity of the two version strings. S_VER_SAME means the two versions are equal, S_VER_NEWER means new_ver is newer than old_ver (or there is no old_ver), and S_VER_OLDER means new_ver is older than old_ver. This method compares two version strings and determines which is older, which is newer, or if they are equal. Note that this method converts only the first three parts of a semantic version string (https://semver.org/).

def dialog(message, buttons, default='', btn_sep='/', msg_fmt='{} [{}]: ')

Expand source code

def dialog(message, buttons, default="", btn_sep="/", msg_fmt="{} [{}]: "):
    """
    Create a dialog-like question and return the result
    
    Args:
        message: The message to display
        buttons: List of single char answers to the question
        default: The button item to return when the user presses Enter at the 
            question (default: "")
        btn_sep: Char to use to separate button items
        msg_fmt: Format string to present message/buttons to the user

    Returns:
        A lowercased string that matches a button (or an empty string if the \
            entered option is not in the button list)

    This method returns the string entered on the command line in response to a
    question. If the entered option does not match any of the buttons, a blank
    string is returned. If you set a default and the option entered is just the
    Return key, the default string will be returned. If no default is present,
    the entered string must match one of the buttons array values. All returned
    values are lowercased. The question will be repeatedly printed to the 
    screen until a valid entry is made.

    Note that if default == "", pressing Enter is not considered a valid entry.
    """

    # make all params lowercase
    buttons = [item.lower() for item in buttons]
    default = default.lower()

    # --------------------------------------------------------------------------

    # if we passes a default
    if default != "":

        # find the default
        if not default in buttons:

            # not found, add at end of buttons
            buttons.append(default)

        # upper case it
        buttons[buttons.index(default)] = default.upper()

    # --------------------------------------------------------------------------

    # add buttons to message
    btns_all = btn_sep.join(buttons)
    str_fmt = msg_fmt.format(message, btns_all)

    # lower everything again for compare
    buttons = [item.lower() for item in buttons]

    # --------------------------------------------------------------------------

    while True:

        # ask the question, get the result
        inp = input(str_fmt)
        inp = inp.lower()

        # # no input (empty)
        if inp == "" and default != "":
            return default

        # input a button
        if inp in buttons:
            return inp

Create a dialog-like question and return the result

Args

message

The message to display

buttons

List of single char answers to the question

default

The button item to return when the user presses Enter at the question (default: "")

btn_sep

Char to use to separate button items

msg_fmt

Format string to present message/buttons to the user

Returns

A lowercased string that matches a button (or an empty string if the entered option is not in the button list) This method returns the string entered on the command line in response to a question. If the entered option does not match any of the buttons, a blank string is returned. If you set a default and the option entered is just the Return key, the default string will be returned. If no default is present, the entered string must match one of the buttons array values. All returned values are lowercased. The question will be repeatedly printed to the screen until a valid entry is made.

Note that if default == "", pressing Enter is not considered a valid entry.

def do_bool(val)

Expand source code

def do_bool(val):
    """
    Convert other values, like integers or strings, to bools

    Args:
        val: The value to convert to a bool

    Returns:
        A boolean value converted from the argument

    Converts integers and strings to boolean values based on the rules.
    """

    # lower all test vals
    rules_true = [item.lower() for item in L_RULES_TRUE]

    # return result
    return str(val).lower() in rules_true

Convert other values, like integers or strings, to bools

Args

val

The value to convert to a bool

Returns

A boolean value converted from the argument Converts integers and strings to boolean values based on the rules.

def dpretty(dict_print, indent_size=4, indent_level=0, label=None)

Expand source code

def dpretty(dict_print, indent_size=4, indent_level=0, label=None):
    """
    Pretty print a dict

    Args:
        dict_print: The dictionary to print
        indent_size: The number of spaces to use for each indent level
        (default: 4)
        indent_level: The number of indent levels to use for this part of the
        print process (default: 0)
        label: The string to use as a label (default: None)

    Returns:
        The formatted string to print

    Raises:
        OSError if the first param is not a dict

    Formats a dictionary nicely so it can be printed to the console.
    """

    # sanity check
    if not isinstance(dict_print, dict):
        raise OSError(S_ERR_NOT_DICT)

    # default out
    out = ""

    # print label
    if label is not None:
        out += label + ": "

    # convert indent_size to string and multiply by indent_level
    indent_str = (" " * indent_size) * (indent_level)

    # items will need an extra indent, since they don't recurse
    indent_str_next = (" " * indent_size) * (indent_level + 1)

    # default result opening brace (no indent in case it is nested and is
    # preceded by a key)
    out += indent_str + "{\n"

    # for each entry
    for k, v in dict_print.items():

        # print the key
        out += indent_str_next + f'"{k}": '

        # if the value is a list
        if isinstance(v, list):

            # recurse the value and increase indent level
            ret = (
                lpretty(
                    v,
                    indent_size=indent_size,
                    indent_level=indent_level + 1,
                    label=None,
                )
                + "\n"
            )
            ret = ret.lstrip()
            out += ret

        # if the value is a dict
        elif isinstance(v, dict):

            # recurse the value and increase indent level
            ret = (
                dpretty(
                    v,
                    indent_size=indent_size,
                    indent_level=indent_level + 1,
                    label=None,
                )
                + "\n"
            )
            ret = ret.lstrip()
            out += ret

        # if it is a single entry (str, int, bool)
        else:

            # print the value, quoting it if it is a string
            if isinstance(v, str):
                out += f'"{v}",\n'
            else:
                out += f"{v},\n"

    # get original indent
    indent_str = (" " * indent_size) * indent_level

    # # add closing bracket
    out += indent_str + "}"

    # return result
    return out

Pretty print a dict

Args

dict_print

The dictionary to print

indent_size

The number of spaces to use for each indent level

(default: 4)

indent_level

The number of indent levels to use for this part of the

print process (default: 0)

label

The string to use as a label (default: None)

Returns

The formatted string to print

Raises

OSError if the first param is not a dict Formats a dictionary nicely so it can be printed to the console.

def fix_globs(dir_start, dict_in)

Expand source code

def fix_globs(dir_start, dict_in):
    """
    Convert items in blacklist to absolute Path objects

    Args:
        dict_in: the dictionary with glob strings

    Returns:
        A dictionary of Path objects representing the globs

    Get absolute paths for all entries in the blacklist.
    """

    # the un-globbed dict to return
    result = {}
    dir_start = Path(dir_start)

    # make a copy and remove path separators in one shot
    # NB: this is mostly for glob support, as globs cannot end in path
    # separators
    for key, val in dict_in.items():
        dict_in[key] = [item.rstrip("/") for item in val]

    # support for absolute/relative/glob
    # NB: adapted from cntree.py

    # for each section of blacklist
    for key, val in dict_in.items():

        # convert all items in list to Path objects
        paths = [Path(item) for item in val]

        # move absolute paths to one list
        abs_paths = [item for item in paths if item.is_absolute()]

        # move relative/glob paths to another list
        other_paths = [item for item in paths if not item.is_absolute()]

        # convert relative/glob paths back to strings
        other_strings = [str(item) for item in other_paths]

        # get glob results as generators
        glob_results = [dir_start.glob(item) for item in other_strings]

        # start with absolutes
        new_val = abs_paths

        # for each generator
        for item in glob_results:
            # add results as whole shebang
            new_val += list(item)

        # set the list as the result list
        result[key] = new_val

    # return the un-globbed dict
    return result

Convert items in blacklist to absolute Path objects

Args

dict_in

the dictionary with glob strings

Returns

A dictionary of Path objects representing the globs Get absolute paths for all entries in the blacklist.

def get_dict_from_file(path_cfg)

Expand source code

def get_dict_from_file(path_cfg):
    """
    Open a json file and return the dict inside

    Args:
        path_cfg: Path to the file containing the dict

    Returns:
        The dict contained in the file

    Opens the specified file and returns the config dict found in it.
    """

    # set conf dict
    try:
        with open(path_cfg, "r", encoding="UTF-8") as a_file:
            return json.load(a_file)

    # file not found
    except FileNotFoundError as e:
        raise OSError(S_ERR_NOT_FOUND.format(path_cfg)) from e

    # not valid json in file
    except json.JSONDecodeError as e:
        raise OSError(S_ERR_NOT_JSON.format(path_cfg)) from e

Open a json file and return the dict inside

Args

path_cfg

Path to the file containing the dict

Returns

The dict contained in the file Opens the specified file and returns the config dict found in it.

def load_dicts(paths, start_dict=None)

Expand source code

def load_dicts(paths, start_dict=None):
    """
    Combines dictionaries from all found paths

    Args:
        paths: The file path or list of file paths to load
        start_dict: The starting dict and final dict after combining (default:
        None)

    Returns:
        The final combined dictionary

    Raises:
        FileNotFoundError: If the file does not exist
        json.JSONDecodeError: If the file is not a valid JSON file

    Load the dictionaries from all files and use combine_dicts to combine them.
    """

    # sanity check
    if isinstance(paths, (str, Path)):
        paths = [paths]

    # set the default result
    if start_dict is None:
        start_dict = {}

    # loop through possible files
    for path in paths:

        # sanity check
        path = Path(path).resolve()

        # sanity check
        if path is None or not path.exists():
            print(S_ERR_NOT_EXIST.format(path))
            continue

        # try each option
        try:

            # open the file
            with open(path, "r", encoding="UTF-8") as a_file:
                # load dict from file
                new_dict = json.load(a_file)

                # combine new dict with previous
                start_dict = combine_dicts([new_dict], start_dict)

        # file not JSON
        except json.JSONDecodeError as e:
            raise OSError(S_ERR_NOT_VALID.format(path)) from e

    # return the final dict
    return start_dict

Combines dictionaries from all found paths

Args

paths

The file path or list of file paths to load

start_dict

The starting dict and final dict after combining (default:

None)

Returns

The final combined dictionary

Raises

FileNotFoundError

If the file does not exist

json.JSONDecodeError

If the file is not a valid JSON file

Load the dictionaries from all files and use combine_dicts to combine them.

def lpretty(list_print, indent_size=4, indent_level=0, label=None)

Expand source code

def lpretty(list_print, indent_size=4, indent_level=0, label=None):
    """
    Pretty print a list

    Args:
        list_print: The list to print
        indent_size: The number of spaces to use for each indent level
        (default: 4)
        indent_level: The number of indent levels to use for this part of the
        print process (default: 0)
        label: The string to use as a label (default: None)

    Returns:
        The formatted string to print

    Raises:
        OSError if the first param is not a list

    Formats a list nicely so it can be printed to the console.
    """

    # sanity check
    if not isinstance(list_print, list):
        raise OSError(S_ERR_NOT_LIST)

    # default out
    out = ""

    # print label
    if label is not None:
        out += label + ": "

    # convert indent_size to string and multiply by indent_level
    indent_str = (" " * indent_size) * (indent_level)

    # items will need an extra indent, since they don't recurse
    indent_str_next = (" " * indent_size) * (indent_level + 1)

    # default result opening brace (no indent in case it is nested and is
    # preceded by a key)
    out += indent_str + "[\n"

    # for each entry
    for v in list_print:

        # if the value is a list
        if isinstance(v, list):

            # recurse the value and increase indent level
            ret = (
                lpretty(
                    v,
                    indent_size=indent_size,
                    indent_level=indent_level + 1,
                    label=None,
                )
                + "\n"
            )
            out += ret

        # if the value is a dict
        elif isinstance(v, dict):

            # recurse the value and increase indent level
            ret = (
                dpretty(
                    v,
                    indent_size=indent_size,
                    indent_level=indent_level + 1,
                    label=None,
                )
                + "\n"
            )
            out += ret

        # if it is a single entry (str, int, bool)
        else:

            # print the value, quoting it if it is a string
            if isinstance(v, str):
                out += indent_str_next + f'"{v}",\n'
            else:
                out += indent_str_next + f"{v},\n"

    # get original indent
    indent_str = (" " * indent_size) * indent_level

    # # add closing bracket
    out += indent_str + "]"

    # return result
    return out

Pretty print a list

Args

list_print

The list to print

indent_size

The number of spaces to use for each indent level

(default: 4)

indent_level

The number of indent levels to use for this part of the

print process (default: 0)

label

The string to use as a label (default: None)

Returns

The formatted string to print

Raises

OSError if the first param is not a list Formats a list nicely so it can be printed to the console.

def pascal_case(a_str)

Expand source code

def pascal_case(a_str):
    """
    Format a string in Pascal case

    Args:
        a_str: A string to convert to Pascal case

    Returns;
        The Pascal cased string

    Formats the given string to a Pascal case equivalent, ie. "my_class"
    becomes "MyClass".
    """

    # do formatting
    name_pascal = a_str
    name_pascal = name_pascal.replace("_", " ")
    name_pascal = name_pascal.replace("-", " ")
    name_pascal = name_pascal.title()
    name_pascal = name_pascal.replace(" ", "")

    # return result
    return name_pascal

Format a string in Pascal case

Args

a_str

A string to convert to Pascal case

Returns; The Pascal cased string

Formats the given string to a Pascal case equivalent, ie. "my_class" becomes "MyClass".

def pp(obj, indent_size=4, label=None)

Expand source code

def pp(obj, indent_size=4, label=None):
    """
    Pretty print a dictionary or list

    Args:
        obj: The dictionary or list to print
        indent_size: The number of spaces to use for each indent level
        (default: 4)
        label: The string to use as a label (default: None)

    Returns:
        The object formatted for printing

    Raises:
        OSError if the first param is not a dict or list

    Formats a dictionary or list nicely and prints it to the console. Note that
    this method includes magic commas in the output, and therefore cannot be
    used to create true JSON-compatible strings. It should only be used for
    debugging.
    """

    # the default result
    result = ""

    # call different pretty functions depending on the object type
    if isinstance(obj, dict):
        result = dpretty(obj, indent_size, 0, label)
    elif isinstance(obj, list):
        result = lpretty(obj, indent_size, 0, label)
    else:
        raise OSError(S_ERR_NOT_DICT_OR_LIST)

    # print the result
    print(result)

Pretty print a dictionary or list

Args

obj

The dictionary or list to print

indent_size

The number of spaces to use for each indent level

(default: 4)

label

The string to use as a label (default: None)

Returns

The object formatted for printing

Raises

OSError if the first param is not a dict or list Formats a dictionary or list nicely and prints it to the console. Note that this method includes magic commas in the output, and therefore cannot be used to create true JSON-compatible strings. It should only be used for debugging.

def save_dict(a_dict, paths)

Expand source code

def save_dict(a_dict, paths):
    """
    Save a dictionary to all paths

    Args:
        a_dict: The dictionary to save to the file
        paths: The path or list of paths to save to

    Raises:
        OSError: If the file does not exist and can't be created

    Save the dictionary to a file at all the specified locations.
    """

    # sanity check
    if isinstance(paths, (str, Path)):
        paths = [paths]

    # loop through possible files
    for path in paths:

        # sanity check
        path = Path(path).resolve()

        # try each option
        try:

            # make sure path is absolute
            if not path.is_absolute():
                print(S_ERR_NOT_CREATE.format(path))
                continue

            # first make dirs
            path.parent.mkdir(parents=True, exist_ok=True)

            # open the file
            with open(path, "w", encoding="UTF-8") as a_file:
                # save dict tp file
                json.dump(a_dict, a_file, indent=4)

        # raise an OS Error
        except OSError as e:
            raise OSError(S_ERR_NOT_CREATE.format(path)) from e

Save a dictionary to all paths

Args

a_dict

The dictionary to save to the file

paths

The path or list of paths to save to

Raises

OSError

If the file does not exist and can't be created

Save the dictionary to a file at all the specified locations.

def sh(cmd, shell=False)

Expand source code

def sh(cmd, shell=False):
    """
    Run a program or command string in the shell

    Args:
        cmd: The command line to run
        shell: If False (the default), run the cmd as one long string. If True,
        split the cmd into separate arguments

    Returns:
        The result of running the command line, as a
        subprocess.CompletedProcess object

    This is just a dumb convenience method to use subprocess with a string
    instead of having to convert a string to an array with shlex every time I
    need to run a shell command.
    """

    # make sure it's a string (sometime pass path object)
    cmd = str(cmd)

    # split the string using shell syntax (smart split/quote)
    # NB: only split if running a file - if running a shell cmd, don't split
    if not shell:
        cmd = shlex.split(cmd)

    # get result of running the shell command or bubble up an error
    try:
        res = subprocess.run(
            # the array of commands produced by shlex.split
            cmd,
            # if check is True, an exception will be raised if the return code
            # is not 0
            # if check is False, no exception is raised but res will be None,
            # meaning you have to test for it in the calling function
            # but that also means you have no information on WHY it failed
            check=True,
            # convert stdout/stderr from bytes to text
            text=True,
            # put stdout/stderr into res
            capture_output=True,
            # whether the call is a file w/ params (False) or a direct shell
            # input (True)
            shell=shell,
        )

    # check if it failed
    except subprocess.CalledProcessError as e:
        raise OSError(S_ERR_SHELL) from e

    # return the result
    return res

Run a program or command string in the shell

Args

cmd

The command line to run

shell

If False (the default), run the cmd as one long string. If True,

split the cmd into separate arguments

Returns

The result of running the command line, as a subprocess.CompletedProcess object This is just a dumb convenience method to use subprocess with a string instead of having to convert a string to an array with shlex every time I need to run a shell command.