Dep Software Build Automation Tool

dep is a software build automation tool, similar in function to make.

It features:

• A simple, powerful language
• Determination of sources and targets by analysing command lines
• Determination of included files by simply reading source files
• Complete integration and handling of subprojects

Overview

dep's language requires little more than the build commands that it must run. For example, this:

cc -o fnord.o -c fnord.c

is a complete build rule in dep. With a careful reading of the command, dep can tell that it:

• writes fnord.o
• reads fnord.c
• reads all of the files that fnord.c includes, along with all of the files that they include, and so on recursively.

The little more that dep needs is the syntax of the command-line options, which is provided by a line such as:

syntax -mixed gcc -g -f: -I:%includedir -W: -o:%out -c -l:%library -L:%libdir -.\* %in

in /etc/depsyntax .

dep provides, but does not require, a syntax for include files. It looks like this:

fileinfo fnord.cpp OK 2022-11-22 12:34:56.789 123456 includes foo.h

(There is a little more information in there, namely a build state, a modification time, and a file size). But that is for a cached copy of the information. Most often, dep simply uses the native include syntax, by reading the source file directly and looking for lines such as:

#include "foo.h"

dep can also easily handle multiple build projects, similar to make's recursive functionality, but without requiring recursion. dep simply includes all of a subprojects rules, and runs them as necessary. Usually they will be in different directories; dep will change the filenames as necessary, and change directory before running any commands found there.

dep also runs its build commands in parallel whenever possible.

DEP vs MAKE

UNIX make, GNU make are successful build tools and GNU make in particular is still being improved. dep, however, is deliberately a break from the past. It is intended to be a successor to make, but not compatible. There are two fundamental reasons for this:

• The make tools require (a) as input sources, targets, and commands, which (b) may be implicitly constructed and (c) may be tweaked. Whereas dep requires (a) just the commands, which (b) may be procedurally constructed and (c) are automatically analysed to determine the source and target dependency relationships.
• Make's fundamental data type is the string, whereas dep's fundamental datatype is the filename. This change allows dep to handle multiple-subproject, multi-directory environments with much more flexibility.

In addition, in comparison to GNU make, there are several things that dep can do that make can't. These are:

• Determine sources and targets by analysis of build commands
• Determine include files by simple reading of the source files
• Directly model include files as include files, instead of as additional source files
• As necessary, compare file contents instead of just assuming that a rebuilt file has changed
• Provide straightforward constructs such as loops and conditions in the source language

There are also things that dep does considerably better:

• Subprojects (i.e. include foo/DepFile) are drawn directly into the project tree instead of using recursive make
• Determination of include files directly from source files (i.e. #include "fnord.h")
• Running commands without invoking a shell
• Handling of filenames that contain whitespace

And there are things that dep simply makes easier:

• Commands with multiple targets
• Parallel execution
• Multiple platforms

There are also some features of GNU make that dep deliberately does not attempt to replicate:

• .INTERMEDIATE, .SECONDARY, .PRECIOUS
• :: syntax
• Multiple build rules for a target
• Reprocessing of variable assignments
• Recursive make
• Jobservers
• VPATH

And there are some features that dep implements quite differently:

• .PHONY . dep splits this into two features:
  1. %pseudo (if its commands should always be run)
  2. %alias (if it has no commands)
• Suffix rules and implicit rules (dep knows these as % pattern rules. They are not present by default, but you can use them if you like)

dep is written in C++ .

Contact

Have you found a bug? Got a suggestion? Maybe it's working perfectly! Let me know how you go.

Email me (Matthew Rickard) at dep@depbuild.org

Release Notes

dep 0.5

dep 0.5 was released on 12 October 2024. Changes are:

• improvements to build ordering via whichComesFirst
• cached getFileState
• added timestampsize
• added implementsHeader
• cached getCompletionSeconds
• refactored shared memory
• runs recently-failed steps until 15% through the job, then switches to critical-path steps
• when looking for the finished job, stopped looking through old jobs
• improved pattern matching of #include statements
• stopped looking for #include statements when the cached copy is OK
• debug senseModTime/senseSize/senseRule/FILEFLAG\_STAT\_CHANGED
• get the updated mod time before writing it back to cache
• Fixed 'file exists' errors on mkdir commands
• Gradually scaled up from number of processors (e.g. 8) number of register sets (e.g. 16)
• Fixed bug that sometimes reset the number of pipelines
• Fixed bug that sometimes let there be more pipelines than register sets
• Fixed syntax lookup of commands starting with "-":w
• Added new acceptance test in Gossammer Markup Language
• Added exists\_file\_ext mechanism that marks all .so/.dll rules at %exists
• Added mechanism that marks all mkdir rules at %exists
• Added --noskip option that runs all %exists rules
• Added mechanism that automatically finds all mkdir dependencies
• Changed %exist to %exists
• Added %alias for precompiled headers
• Added --noreadcache, --nowritecache and --writecache
• Debugged pipeline commands
• Refactored verbosity options and output
• Added --tracefile option
• Stopped making duplicate stat() calls
• Debugged marking of re-generated files
• Debugged reading of #include files from .c files
• Added % patterns
• Shrank some struct sizes
• Fixed memory leaks
• Refactored parser buffer creation+deletion
• Changed --simplesort to --fixedorder
• Refactored nextCommand to remove startMore option
• The set of --treeincludes include files now depends on the list of -I options

dep 0.4

dep 0.4 was released on 30 May 2023. Changes are:

• Added filename and line number to duplicate rule warning messages
• Fixed bug that caused non-duplicate rules to be listed as warnings
• Separated environment variable spaces for parallel execution streams
• Fixed bug that wrote duplicate rules to .DepFileCache
• Fixed bug in determining locations of included source files
• Fixed bug in normalizing file path names

dep 0.3

dep 0.3 was released on 9 March 2023. Changes are:

• Changed findIncludes mechanism to look at filenames instead of options such as -c, to handle instances such as gcc a.c b.o
• Fixed bugs in quote mark parsing
• Combined operations on rule and tree includes

dep 0.2

dep 0.2 was released on 16 February 2023. Changes are:

• Added sensing and invalidation of some misbuilt files (generally 0 byte files, but 8 bytes for .a files)
• Added %outz for files that may validly be zero length
• Fixed bugs in handling of rules with no inputs e.g. mkdir commands
• Added existence dependencies
• Changed -p option to -j
• No need to put " > %out " in syntax lines any more, it is now always assumed
• Fixed bugs in interpretation of nested for loops
• Allowed braces on subsequent lines of for loops
• Changed "if ... endif" to "if { }"
• Fixed some errors in the generation of .DepFileCache and suppressed some others in readback
• Improved line and column number reporting in DepFile syntax error messages
• Added /etc/depsyntax file, which includes syntaxes of common commands
• Fixed bug in export statements within rules: they are now run during execution, not during parsing

dep 0.1

dep 0.1 was released on 7 December 2022.

Copyright and Licence

Copyright (C) 2022 Matthew Rickard

dep is free software licenced under the GNU General Public Licence version 3.