Author:         Jan Nijtmans <jan.nijtmans@gmail.com>
State:          Final
Type:           Project
Vote:           Done
Created:        29-Dec-2017
Post-History:
Keywords:       tcl
Tcl-Version:    9.0
Tcl-Branch:     memory-API
Vote-Results:   3/0/2 accepted
Votes-For:      DKF, JN, KBK
Votes-Against:  none
Votes-Present:  JD, SL

Abstract

This TIP describes the non-controversial part of the Tcl 9 changes: Make Tcl 9 ready for the 64-bit era.

Rationale

int is the type of many Tcl API function arguments and return values, and also of many struct fields, but it doesn't provide sufficient room on 64-bit platforms.

Proposal

For hash functions, replace unsigned int with size_t as the type of the hash value, which allows hash tables to grow beyond 4Gb on 64-bit platforms.
Make size_t the type of all struct fields representing reference counts or epochs.
Change the type of the size parameter of each memory-related function, e.g. Tcl_Alloc(), from int to size_t and change the char * argument to void *.
Change the type of the size parameter of many functions from int (but NOT int *) to Tcl_Size.
Change the type of all ClientData arguments void *. This is actually the same type, but it prevents the need for type casts in some situations.
Provide two new macros, TCL_IO_FAILURE and TCL_AUTO_LENGTH, each defined as ((Tcl_Size)-1), to help extensions use the full 64-bit range with Tcl 9, but still compile with Tcl 8 as well (see below). Provide those macros in Tcl 8.7 too, but defined as (-1).
Modify Tcl_Alloc(), Tcl_Free(), and related functions to redirect to their debugging variants if TCL_MEM_DEBUG is defined: Tcl_Alloc() becomes the same as ckalloc(), ending the general confusion regarding the difference between those two groups of functions: Starting with Tcl 9.0 there no longer any difference. ckalloc() and related functions are deprecated starting with 9.0, but there are no plans to remove them, and no deprecation warning will be given if extensions use them.

On 32-bit platforms this is all 100% upwards binary compatible provided no internal API is used, since some internal structs might have incompatible but externally invisible changes.

On 64-bit platforms, those changes cause binary incompatibility. Therefore the TCL_STUB_MAGIC value needs to change, so extensions compiled using Tcl 9 headers will not load in Tcl 8 and reverse.

Addendum

After TIP #660 was accepted, a lot of functions changed from using size_t to ptrdiff_t parameters. In order to prevent confusion, this change has been adapted in the TIP text above as well. Also -DTCL_8_COMPAT is not provided any more, because it is no longer needed.

Implementation

The implementation of this TIP can be found in the memory-API branch.

Copyright