UI21DEB - deBUGGER ================== About ----- UI21DEB is a DOS real mode deBUGGER focusing on file I/O activity performed via INT $21. While the world is already full debuggers ("Insight" real mode debugger by Oleg & others, FreeDOS DEBUG, G[R]DB, "deb" by Japheth, OlyDbgr (does it really work in DOS ?), WATCOM's "WD" (seems not to work in DOS either ?), "SD" by Narech Koumar (dropped with DOS/32A version 9.xx ?), and last but not least the famous FTE-debugger), UI21DEB has some unique features, OTOH it also lacks some features known from the other debuggers, so NO, it doesn't attempt to obsolete them all, see "Comparison" below for details. While a debugger is usually a tool for developers and advanced users only, UI21DEB has some features making it useful to "average" users as well, see "For users" below. Package contents ---------------- # UI21DEB.COM (6'644 Bytes, 060F0FB6451F9F7C7DE04FCD2569E33C) main file, DOS ".COM" TSR executable # UI21DEB.ASM (136'526 Bytes, 2F013DB32500CF5B96CAE89300E5F383) source code # UI21DEB.TXT (21'555 Bytes, ????????????????????????????????) this text # UIEXAMPL.TXT (92'200 Bytes, 09489E4B3596D2793C09E3B06018F9FD) examples of deBUGGER output: detailed list of calls, with comments on BUG's and oddnesses encountered # (5 files, total 28'857 Bytes) BUGYWGET.PNG 8'231 7A9A7F7D3DD89CDB88EBD90479C4218C FAT28KIL.PNG 4'745 F8B8EDC38E057CF772BA74BA809299A5 GIFENONT.PNG 4'407 318D229E1BC2DEF1311C993FCAE5207C KO9HANG.PNG 7'785 D1D5BE1FF4C458F5298A12D18EC3F54C MTRRONNT.PNG 3'689 2471725CB5610EB7563EFD92493BB0B3 screenshots (KO9HANG done with BOCHS, other with SNARF) System requirements ------------------- For running the deBUGGER: # CPU: At least 8086 or compatible # RAM: At least 512 KiB # Video: any with text mode support # Storage: some 100 KiB of free space # OS: My new (not yet released) DOS, RxDOS 7.30, FreeDOS 1.1, EDR-DOS 7.01.08 or compatible For viewing the saved debug buffers: # A file viewer supporting both text and "hex" mode (appropriate for given hardware, Japheth's "S" is a good choice, requires 80386 CPU, to be fixed later) # Video: any (preferably with 80x50 chars text mode supported) For compiling the deBUGGER from the source: # CPU: At least 80386 or compatible # RAM: A few MiB (check it out) # Tools: FASM 1.67.29 or better For viewing the screenshots: # Viewer with PNG format support # Hardware able to run it and supporting at least 640x480x4bpp graphics General features ---------------- # Debugging INT $21 calls, see below "Debug" # Fixing faulty filenames, see below "Filenames" # Fixing / workarounding various suboptimal and faulty behaviour arising from interaction of DOS application with the kernel, tending to cause unproductive controversy or even fight about who is responsible for bug: whether the kernel developer or the application developer, or some "faulty but deliberate and documented" behaviour, see below "Misc features" for details # Can be used with both real mode applications as well as various types of protected mode DOS applications General limitations ------------------- # Can't be used with low memory hogs as it hogs 63.75 KiB itself (to be fixed) # No other TSR should touch INT $21 while UI21DEB is enabled, see "I" switch below Debug ----- #### Features: # Can register INT $21 calls # Can "kill" , "fake" and "fail" INT $21 calls # Provides 3 outputs: count of selected groups of calls, a detailed list of selected calls including parameters passed and results, and an overall call statistics (registering all calls, almost impossible to overflow) # Excellently suitable to search for bugs and performance killers in file I/O, like redundant repeated calls, spurious seeks, faulty filenames, unreasonably many read or write calls with negligible amount of data, excessive attribute peeking or writing, notorious attempts to use NTLFN calls, spurious file I/O at all, ... # Can dicover all files that an applicaton is trying to load / save / use / access, including temporary files deleted before termination, DLL's loaded "dynamically" using "LoadLibrary" invisible in the import list, and undocumented ".INI" or similar files # Very useful for debugging virii (DOS is safe and this deBUGGER makes it even safer) # NTV-DM emulation in DOS (genuine NT not needed anymore) # Useful for checking application compatibility with both the future and the past # Can debug real mode applications and compilers they were brewed with # Can debug DGJPP applications as well as DGJPP itself # Can debug non-DGJPP DPMI applications and the compilers involved in creation of them # Can debug Win32 applications running in DOS using HX Extender as well as HX itself (see below for limitations) # Can disable FAT28 AH=$73 calls, thus temporarily emulate an old DOS version # Can disable the "NTLFN API" AH=$71 calls, (including King Udo's "looooooong seek" AX=$7142 call), this feature turned out to be very useful to "fix" experimental very incomplete NTLFN support in some EDR-DOS "WIP" versions # Can disable the "extended open" AH=$6C call (this breaks many things) # Can disable obsolete FCB calls (except AH=$29) # Can separately disable the "parse form FCB" AH=$29 call (this breaks EDR-DOS) # Can disable floppy calls # Can set the famous "NTrue DOS version" to $3205 (this shouldn't be able to break any DOS application, however, if anyone bothers to check oneself, tests will bring up a massive evidence of the opposite quickly) # Reports names of applications causing the calls (not well with some DOS entenders, see below) # Can selectively debug one application only or one application and everything spawned from it # Can detect that someone else hacked INT $21 (see "I" switch below) #### Limitations: # Can't debug problems that cause the PC to hang or TripleFault (use an emulator like BOCHS) # Can't set breakpoints or single-step code (see list of other debuggers above), also no disassembler (see above + BIEW + NDISASM + + DISTORM) and no assembler (try FASM or NASM or so) # Debugging of Win32 applications isn't optimal, since UI21DEB runs "below" DKRNL32.DLL and HDPMI32, which will split read and write requests of more than cca 64 KiB, as well as workaround GetFileSize[Ex] calls with 3 seeks and SetFilePointer[Ex] with a negative distance by 2 seeks, and even worse workaround GetFileInformationByHadnle by "directly" peeking the SFT, completely bypassing INT $21 and the deBUGGER, at least, after having tried INT $21 / AX=$716C before (registered by UI21DEB). So a debug version of DKNRL32.DLL might be a better choice here. # You can't single-step into the DOS kernel while UI21DEB is active, disable or uninstall it in such a case # Some applications, most notably the DOS/32A extender, do temporarily hack INT $21 for their "internal purposes", to be able to use them with UI21DEB the "I1" option must be set, and such "internal" calls won't be visible to UI21DEB anymore. # If "I2" is used there is still no write-sensitive breakpoint on INT $21, thus the incident will get noticed at occasion of earliest INT $21 call chained to UI21DEB, not immediately. # Some applications do use an outsourced DOS extender and run then under the name of the extender, not its own. To get around this in debug modes 3 and 4, include the extender into the application main file if possible, or set the selectivity name to the one of the extender, not the application. Even worse, DPMILD32 from HX hacks the PSP effectively getting "anonymous" then. # Reloading UI21DEB itself while an other instance is already active with debug hack on leaves 1 junk entry in the call list buffer (self-debugging) Filenames --------- # Can fix (to some degree) faulty filenames ("multiple-slash-BUG" and "multiple-dot-BUG") supplied by many (including, but not necessarily limited to) NTVDM-DGJPP applications, avoiding duplicates and "ENOENT" , allowing to run them in DOS # Fixing strategy of UI21DEB doesn't cause any filesystem pollution, slowdown or risk of corruption # Names "only" longer than 8 chars like "VGABIOS-cirrus" and "VGABIOS-cumulus" are currently not being fixed and will still duplicate # Doesn't provide 100% NTFLN compatibility, while it's verified to be sufficient for many cases (GIFSICLE, WGET, 7-ZIP, OPTIPNG), there could be applications not happy with it, so things like STARLFN or DOSLFN might still have a chance then Misc features ------------- # Disable "AbortRetryFail" (FreeCOM has an "/F" switch doing the same more easily, EDR-DOS hasn't) # Disable "Remove diskette from A: and insert it into B:" (no known alternatives) # Faster floppy failure - reduces unreasonably many attempts to read from a (not ready) floppy to just 1, if an access fails because of of devide not ready or physical media error, any further attempts within 8 seconds are failed immediately and do restart the timer # Can completely disable floppy accesses, thus saving time and drive live time Usage ----- UI21DEB is a DOS TSR, it installs itself at the very earliest run, subsequent runs will just control the instance already present in memory. Multiple different versions should not be used at same time, as they would chain, not update. The TSR of course can be uninstalled, but only if no other TSR hacked INT $21 later, and idle reentrancy depth is ZERO. The installed TSR can be enabled (redirects INT $21), ir disabled, then UI21DEB is completely passive and safe, and can't cause any evil except hogging memory. You should NOT install any other TSR redirecting INT $21 after UI21DEB, this causes always loosing the control over it, blocks uninstallation and by default switches the active TSR into a "safe mode" disabling all hacks. However if the other TSR correctly disables or uninstalles itself, UI21DEB becomes fully accessible again. Running it with no options will report existing status, disable all hacks, but not clear buffer, unless requested or at a new install. All the available hacks are independent, any mix of them (with a few exceptions, see below) can be activated. If the "debug" hack is used together with some disabling hack, the disabled calls still will be processed and registered by UI21DEB, but not executed by original INT $21 code. #### Options: Disabling / faking hacks (9): # 3 - disable AH=$73 FAT28 calls # N - disable AH=$71 NTLFN calls # E - disable "extended open" AH=$6C call # F - disable FCB calls (except AH=$29) # 9 - disable the AH=$29 "parse form FCB" call # P - disable floppy (both "A:" and "B:") # Q - faster floppy failure (both "A:" and "B:" , and independent) # A - disable "AbortRetryFail" and "Remove diskette from A: insert diskette into B:" # V? - enable NTV-DM emulation ("?" doesn't work yet) Note: "P" excludes "Q" and "Q" includes "A". Enabling hacks (2): # X? - enable filename fixing ("?" doesn't work yet) # D?,? - enable debug (3 lists, "?,?" doesn't fully work yet, use "D1" to "D4" to select method, default "D" is "D2") Buffer control: # C - clear buffers (delete stuff collected by "D" option) # S - save and clear buffers Overall control: # I - defines how to deal with someone else hacking INT $21: 0 (default) - "go safe", 1 - ignore (and miss calls), 2 - hang with fatal error "I" Uninstall: # KO - don't combine with "S" Debug modes + options (not yet), call registering, format of buffers -------------------------------------------------------------------- #### Debug modes: 1. Insensitive to application name, don't poll PSP 2. Insensitive to application name, do poll PSP (default) 3. Sensitive to application name, do poll PSP, debug given application only 4. Sensitive to application name, do poll PSP, debug given application as well as everything spawned from it For [3] and [4] a name must be given, can be anywhere in the commandline, but must be quoted, if none given UI21DEB drops back to mode [2]. The "D" option enables the debug hack - the core feature of UI21DEB. 3 output buffers (1 of $50 Bytes and 2 per $7000 Bytes) are serviced, UI21DEB will count and "collect" calls (try to, until a saturation arises) into all 3 at same time, but store the call data differently. The group couting buffer counts calls matching application selectivity (if selective mode used) and given group (except the "all" amount), it contains 18 (not yet fully functional) UINT32 values, giving amount of: # All calls (optionally only by given application) # FCB calls (except AH=$29) # Seek calls (AH=$42 or AX=$7142) # Rename calls (AH=$56 or AX=$7156) # Free space calls # Input string SFN calls [-] # Input string LFN calls [-] # Output string calls [-] # NTLFN calls (AH=$71, excluding AX=$7142 and AX=$71A6) # FAT28 calls (AH=$73) # File open and create calls (AH=$3C $3D $5B $6C and AX=$716C) # Floppy calls # Amount of read AH=$3F calls [-] # Total amount of data read (supposed to) [-] # Amount of write AH=$40 calls [-] # Total amount of data written (supposed to) [-] # [-] # [-] The detailed list buffer tries to collect "general" obligatory information and calls as well as optional selected calls, one after other as they arrive, and includes additional information like register contents and strings. This approach of course produces much more data and brings a risk of early buffer saturation, in such a case no further calls are registered here anymore, still the other 2 buffers will continue processing them independently. Following calls are or can be collected: - (obligatory) AH=0 and AH=$4C exit calls, AH=$31 go TSR, AH=$4B spawn, AH=$50 SetPSP, rename calls - (quasi-obligatory) One call after every PSP change - (enabled) Calls having an input string, SFN or LFN - (enabled) Calls having an output string (string not visible yet) - (disabled) Close & Read & Write calls - (enabled) Seek calls, both AH=$42 and AX=$7142 - (enabled) GetDiskFreeSpace calls - (enabled) Version calls - (enabled) Calls getting killed/faked/failed This buffer is designed to debug the most popular calls causing trouble not because they are being used at all, but because they are used wrongly (garbage string, bad numeric values), unreasonably frequently, or at wrong time. Note that obligatory and quasi-obligatory calls will always get registered, even if selective debugging is used and application doesn't match. The overall stat buffer contains a list of all "different" calls issued (optionally only by given application), classified and sorted by AX value (with tweaks, see below), every entry is 8 bytes in size: - 2 Bytes: $0F,$0B - 2 Bytes: AX value of call (see below) - 4 Bytes: Amount of such calls The buffer can store $0E00 = #3'584 "different" calls, if this amount is reached, no calls are being added anymore, nevertheless additional occurrences of already "known" calls are still processed. Of one "call type" up to 2^32-1 occurrences can be counted, then a saturation arises (possible at all ???), no wrap, and other calls are still counted, and even "new" calls can be added if the amount of "different" calls hasn't saturated yet. This makes this buffer useful to debug unknown / wild / non-standard calls, it will show what call or that such a call occurred, and how many times. Some calls are "tweaked" to reduce "pollution" by random AL values and make the stat more useful: # AL is ZERO'ized for following calls not using it: - 2&9 (screen writing) - $19 (get default "driver") - $1A (set D.T.A.) - $2C&$2D (get/set time with 1/100s precision, believe or not) - $2F (get D.T.A.) - see below about $36 - $38...$3C (nationalism, MD, RD, CWD, create) - NOT $3D (has access and sharing modes, see #01402) - $3E (close) - see below about $3F and $40 - $41 (delete) - NOT $42 (has origin type) - see below about $4E - $4F ("FindNext") - $56 (rename) - $5B (create, also) # AL is set to a useful value: - For AH=$36 "GetDiskFreeSpaceObsolete", AL is set to DL (volume letter) - For AH=$3F (read) and AH=$40 (write), AL is set to CH - stores amount of data transferred in 256-Bytes units - For AH=$47 "GetCurrentDirectory", AL is set to DL (volume letter) - For AH=$4E "FindFirst", AL is set to CL (attribs) The tweaks affect of course the stat buffer only, not the detailed list, and even less the AX value passed further to original INT $21 code. Saving debug results -------------------- The "S" option saves the 3 buffers into a file named preferably "UI21DEB.DAT". If this name is already occupied by an existing file, UI21DEB will not overwrite it, but try to find a free name. The buffers will be cleared in any case, even if saving fails. The file size is $E080 = #57'472 Bytes and it contains UI21DEB's control area ($30 = #48 Bytes), the group counting buffer ($50 = #80 Bytes), detailed call list buffer (at $80, up to $7000 Bytes) and the overall call stat buffer (at $7080, up to $7000 Bytes). Examples -------- Running an application "BUGGYAPP" suffering from the "multiple-slash-BUG" or "multiple-dot-BUG" : UI21DEB X BUGGYAPP UI21DEB The "Faulty Filename Fix" is pretty safe and seems not to break anything if left enabled, still, there is a small risk that it could. Debugging an application "SUSP" : UI21DEB C D SUSP UI21DEB S Again, should be safe if left enabled. Also, the "C" option is only useful if buffer was not already empty before. NTV-DM emulation for "NTLOVER" application : UI21DEB V NTLOVER UI21DEB Make sure, when not needed anymore, to disable this hack ASAP since it really breaks many things. FAT28 disabling for "LEGACYAP" application : UI21DEB 3 LEGACYAP This way you can emulate MS-DOG on FreeDOS or old [Open-]DR-DOS on EDR-DOS and check whether and how well applications still run. Selective debugging of "COMP.EXE" (some compiler), including stuff spawned from it, like the assembler and linker, disabling NTLFN calls, and setting "I1" to allow some DOS extender to hack INT $21: UI21DEB C D4 "COMP" N I1 DEL HELLO.* EDIT HELLO.WOW COMP HELLO.WOW HELLO UI21DEB S Deleting, editing and testing "HELLO.EXE" won't get registered here. See UIEXAMPL.TXT file for more examples and detailed info. Comparison ---------- There are or used to be a few other applications registering INT $21 calls, one of them is XRAY, payware from 1990, abandoned : + XRAY: additinally intercepts INT $13 trying to count sectors and inspect DOS buffering + XRAY: additinally tries to measure time consumed by the call + XRAY: more verbose list + XRAY: can send data to a printer (risky, though) - XRAY: not free nor open source - XRAY: doesn't recognize any calls with AH > $68, pretty useless with "modern" NTFLN-aware "DOS" applications, even "extended open" AH=$6C is getting missed - XRAY: INT $13 evaluation understands CHS calls only, making it pretty useless now - XRAY: can send data to the screen, a file or printer, but not collect it in memory, much more obstrusive - XRAY: no high capacity statistics - XRAY: no selective debugging - XRAY: lacks all the additional features of UI21DEB For users --------- Some features of UI21DEB can be useful for users as well, most notably: # "X" hack (faulty finename fixing) # "N" hack (disable some incomplete or buggy NTLFN support) # "A" hack (disable annoying "AbortRetryFail" & Co) # "P" hack (disable floppy) # "Q" hack (faster floppy failure) Fatal errors ------------ During its operation, including both TSR install & uninstall & control as well as activity of the TSR code resulting from INT $21 usage, UI21DEB recognizes several possible errors that are considered as fatal and make it necessary to reboot if a such occurs. The type of error is indicated by a character filling the top 3 lines of the screen: M: MCB chain corrupt K: Kick out or unhog environment failed 9: Reentrancy depth attempted to exceed 8 0: Reentrancy depth attempted to sink below ZERO R: Illegal attempt to reenter while INT $21 / AH=$51 in debug modes 2, 3, 4 S: Sigi dead / stack overflow I: Someone else hacked INT $21 after "I2" was set While error "9" can be easily produced by unreasonably deep spawning (see shot), and error "I" by setting "I2" and launching an offending application, all other errors shouldn't really happen, they would indicate memory corruption caused by a severe BUG in UI21DEB, DOS kernel or something else. Copyleft & support ------------------ The UI21DEB deBUGGER is Free, Open Source, and Public Domain. (CL) 2009-05-02 by DOS386 Public Domain - Abuse at your own risk Support forum: "http://board.flatassembler.net/topic.php?t=9736" At the time of writing, the forum is accessible with both Arachne and DosLynx, registration is obligatory but possible to pass without extraordinary capabilities or efforts. EOD ===