Project-Wide Unified TODO List / Schedule

TODO List for TorX Library:

High Priority / Security Audit:

• Rewrite anywhere that we pass n+i to pass n+struct instead. Array of struct works fine for everything except messages.
• Messages should be a linked list, then there is no realloc being called when loading and deleting messages, making deletion of stream messages low-cost.
• UI message will have to register its' location with a void pointer to the library message, and library message will have to have a callback trigger to tell UI to delete the UI message before deleting itself.
• Printing messages by linked list should be possible because we already do it in groups.
• To consider: use reference counter on group stream messages and delete when too old or all sent.
• Every library function should do basic sanity checks (ex: n > -1 && n < threadsafe_read_int(&mutex_global_variable,&max_peer), g > -1 && g < threadsafe_read_int(&mutex_global_variable,&max_group), etc), either with -1 or 0 error level, as appropriate. Diagnosing a -1 shutdown is far easier than a segfault on Android.
• ENUM_PROTOCOL_FILE_INFO_REQUEST maybe should not be facilitated in public groups.
• Certain types of group streams (A/V call data, for example) need only be sent to a subset of group peers.
• Pointer objects are not properly accessible through getter_array and will fail getter_array_sanity_check and getter_group_array_sanity_check, with the exception of message. We will need to implement null checks and strlen.
• offer/request_list is not accessible through getter/setter functions. If we don't rapidly remedy this, we will inconvenience future UI devs.
• Some file related protocols are saved even if logging is disabled. This should be resolved.
• message_resend hasn't been tested in a while. Test with public groups, private groups(signed), p2p.
• Restarting Tor calls sql_populate_peer again. Much of it may be unnecessary if(messages_loaded). load_onion(n) and peer_init or whatever does tor calls may be necessary. More importantly, we should probably find a way to take down onions before shutting down Tor. If we did that, we could eliminate issues.
• GROUP_OFFER_ACCEPT: ONLY respond IF we verify that we already sent this peer an UNUSED invitation (perhaps we should save this with extraneous), otherwise could be issues (ie, a malicious peer could accept multiple times and bring in a bunch of people)
• iOS (Orbot) / System Tor: We need to re-write tor_call so that all calls are on a single connection and ADD_ONION does not utilize "Detached". See Tor's control-spec.txt. That will enable us to run safely without control_password. However it will not achieve Orbot compatibility at least on Android (which starts up with a random CTRL_PORT). We could work with their devs to get them to resolve the issue.
• Tor not dying keeps onions online after crash. Consider the viability of a Tor Killer process that will kill Tor if TorX's PID dies. It could just be a wait() kill() process, being forked from TorX with possession of both Tor and TorX PIDs. This would be useful to ensure that Tor and its hidden services aren't left hanging if TorX crashes or is improperly closed. NOTE: probably better to use just not use detached? Next startup would kill tor via PID.
• Consider the viability of (or move towards) full-duplex messages of all types, and treating files nearly the same as any other message.
• Every message would need to have a checksum (for matching) and a start point, similar to how we do files now.
• File transfer checksums fail after combination of sender crashes, stickers, messages, multi-file transfers, etc. Need audit.
• Callback for invalid or incomplete message (which could be illict attempts to ping or do traffic analysis).
• Minor notice for incomplete, major for invalid + counter.
• Warning: After a file is cancelled, a peer can keep sending data or re-start sending data and we receive/disgard it. This could be exploited to monitor a peer.
• Need to somehow notify a peer that such packets are still inbound.
• Issue: Incomplete messages come in frequently in normal operation (faster than a monitor's framerate)
• Issue: Invalid messages could come *incredibly* infrequently, if a malicious peer (ie, a peer could report a 1000 GB message and send it over weeks, never triggering an 'invalid' warning).
• Thought: Invalid messages already report errors, so do we *really* need redundant callback, which has very little legitimate value?
• Arti / Vanguards
• Need to migrate to Arti as soon as viable and enable full vanguard support https://spec.torproject.org/vanguards-spec/full-vanguards.html
• Background: https://blog.torproject.org/tor-is-still-safe/index.html https://blog.torproject.org/announcing-vanguards-for-arti/
• Alternative mitigation method offering full-vanguards (Note: abandonware) https://github.com/mikeperry-tor/vanguards
• Tor >= 0.4.7.0 already has built-in support for Vanguards-lite: https://spec.torproject.org/vanguards-spec/vanguards-lite.html
• Need a red team building malicious or buggy client/library, which will send undersized/oversized messages, spam attacks, etc.

Medium Priority:

• (Easy fix) In sql_populate_message, if(reverse) and if(protocol == ENUM_PROTOCOL_FILE_REQUEST), we should probably avoid "status = " changes to _ACCEPTED.
• (Easy fix) Implement a "min_fail" to reduce the cost of begin_cascade and peer_offline. It should initialize at 0, be increased to max_i by begin_cascade, and be reduced to min_i by load_more. It should only be utilized by begin_cascade and perhaps peer_offline. Alternative: pass i of last sent message to begin_cascade, but that won't account for load_more, which may or may not be desirable. Simpler though. NO ---> Implement an `int* queue` in peer struct. Add 'i' to it when "Refusing to send_prep" and "Send_prep too early" in send_prep, and when `].stat = ` in load_messages_struc. Read from queue to determine next message to send, in begin_cascade. Remove from queue in packet_removal and when message is deleted (Note: may not be necessary when deleted). Note: This is 'low priority' because show_log_messages limits the amount of messages required to be parsed without a queue. NOTE: Could result in rare double-sends due to race between begin_cascade and packet_removal? Our current setup is subject to the same potential race? Presumably yes on socket_swappable messages, since ENUM_MESSAGE_SENT is set in packet_removal. Both sockets *could* hypothetically send the same message. Solution is to "claim" it somehow? Minor problem, solve later as seperate TODO.
• 2025/04/13 Flutter: "Cannot open file path /data/user/0/com.torx.chat/cache/file_picker/1744539315957/新建文本文档 (2).txt for sending. Check permissions.". Caused by a file being shared from "Recent", then (an hour or two later) accepted by peer. Temporary solution (and good idea): Peer will naturally re-send file with same hash after this bug. When receiving a file offer for a file we already have and have already made a request for (presumably which is stalled), re-request it.
• ENUM_PROTOCOL_FILE_REQUEST is logged so that we can set the status of the file. This is a lot of bloat, considering that one file could have many requests, especially in groups. It would be more efficient (disk space) to store the file status in another manner.
• Group file transfers of 4 byte files fail in file_init: "Coding or IO error. File size != sum of sections" 4!=8. More testing and diagnosis required.
• "Last transferred > transferred" occurs frequently (on the sender side only?) when starting file transfers.
• Blacklisting should only occur if you requested the whole section from a single peer, otherwise we have no idea who we got the bad packet from. We currently have no way of tracking this because we only track split_status_req. We will have to track split_status_req_start and split_status_req_end.
• Zero_ functions should have UI callbacks. There are limitations related to locks regarding how we could implement this. Remember to put them outside the function (after is preferable, but before is fine during shutdown).
• Group File Transfers:
• Important: select_peer should probably return a list of peers, or at least a fd_type and the proposed section.
• select_peer seems to select the original offerer of a file most of the time, rather than utilizing those who sent FILE_OFFER_PARTIAL data.
• We rely on section_update to request the next section when one section is complete, but what if we requested less than a full section from a peer? ie: if we requested from a _PARTIAL? Until we resolve this, we should perhaps avoid requesting partial sections.
• Need a ENUM_PROTOCOL_FILE_REFUSE protocol so that, for example, if a peer has moved/deleted a file or changed its permissions, the requesting peer doesn't wait endlessly for data to arrive after making a request. Note: Can simply send a _PARTIAL with 0/0/0 and have the receiver cancel any requests from this peer.
• CMakeLists.txt:
• Needs to have install directive (LIBRARY DESTINATION lib), to allow independent builds and installs of .so file, without re-compiling UIs.
• set(SSL_ARCH linux-x86_64) is hardcoded, except on android. will be issues on *pi, postmarketOS, macbooks, etc.
• CMakeLists.txt files can be ran through AI. Add debug messages with message(STATUS hello) and run with cmake --trace for debugging.
• Prioritize our own broadcasts: Calling broadcast_add(-1,...), to add a group, could take a very long time because manually added groups are not prioritized in broadcast_threaded. Should probably have an additional queue of high priority (our own) broadcasts.
• Due to message_prep existing within message_send, and only one g being passed around (no distinction between "relevant g" and "target g"), it's impossible to send a group offer into a group.
• Search (again) for ]; and ] = { then ensure zero'd.
• Streaming a group message is going to be a message struct bloat issue because, unlike other stream messages, upon outbound send it is not deleted from group_ctrl (the group_ctrl n,i is not zero_i'd). The solution is to somehow track when all the attempts are done (perhaps in message_send) and then zero_i it (see: sfaoij2309fjfw). We may also need to have a time-deletion where after a certain number of seconds we just consider it no longer relevant.
• Multi-Device / Blinded keys. Use the nonce method to generate blinded keys. Do it using onion_gen.c Upload it using HSPOST. See src/feature/hs/hs_common.c for creation of blinded keys. See any mentions of "blind" "period" "time" in the src. I think HSPOST is somehow combined with ADD_ONION to publish? Unsure and will need to ask #tor-dev
• GROUP FILE TRANSFER Documentation: Outbound uses peer_n fd for disk reads, Inbound uses group_ctrl fd for disk writes. When receiving file request, it appears we store outbound date in the group_peer_n. This needs to be documented/understood. When changing file status in process_pause_cancel, we might need to iterate through the relevant group_peer_n and set their status too? idk if we store status there or only transfer amount.
• We can't support uint64_t file sizes because ftell/fseek only support long int. Unsure what to do about this. It affects our file transfer protocols.
• accept_automatically. Can either be a toggle or a size slider. size_t is probably most logical. Could also have a list of accepted filetypes. NOTE: Enabling this option will require a check for download_dir. Also, this option should could be DANGEROUS for insecure platforms like Windows/OSX/iOS/Android, where CSAM scanning exists.
• If a split_folder exists, files inside should be labelled via truncated checksum rather than filename.
• file_set_path should be able to move a file, if existing and destination is writable.
• Hash is written to disk in plaintext for our .split file. Perhaps we should encrypt (no) or salt + rehash it (not all hashes, just this hash written to .split file). The reason we don't is because the file itself is written next to it, so what is the security in having the hash salted? We do attempt to securely delete .split files.
• Sodium memory management overhead: Check for compile-time flags that could disable the wasted pages except on debug builds. sodium_malloc() wastes too much memory by allocating 4kb of space for each malloc. See HAVE_PAGE_PROTECTION
• Clean up error messages / notices / checkpoints using macro defined levels to determine whether debug level + whether they contain sensitive info.
• Error messages cannot be inside torx_read/torx_write in case callbacks are guaranteed to be handled asyncronously, by adding them to a queue. Currently, very few are, if any.
• 2025/04/29: After dozens or hundreds of disconnections occuring rapidly, we get "ERROR MEMORY sqlcipher_mlock: mlock() returned -1 errno=12". This is possibly from too many sodium_malloc not being free'd when disconnecting?

Low Priority:

• A "getter_exists" function could exist, which would at minimum be used for pointers to see if they are NULL. Could error out if not a pointer, or do some other form of checks like a zero check on the first byte (for onions, peeronions) or is_null (for hashes, other arrays).
• Change 'n' to 'file_n' in file related functions that are passed n,f pairs or n,f,peer_n. n and file_n must NEVER be used together in a function, to prevent confusion/mistakes. Relevant variable names: file_n, target_n, group_n, peer_n, etc.
• If messages.db is deleted or message history is deleted, _ENTRY_REQUEST _FAIL messages will be deleted, which means public groups we are trying to join will never be joined.
• 2025/01/15: Pausing is near indistinguishable from simply being inactive (ENUM_FILE_INACTIVE_ACCEPTED). As long as file_path is set, we treat it as accepted, and then any incoming offer will start the transfer. This inability to permanently pause (without cancelling) is annoying, but we never had a strong pause because either peer could unpause. Therefore, this is OK. Do not abolish pause protocol. It serves other purposes.
• We have some places where we unnecessarily fwrite multiple times in sequence rather than once, such as in initialize_split_info.
• Investigate changing build system to Meson and using Git submodules, as suggested in Issue #2.
• Don't send a ENUM_PROTOCOL_FILE_OFFER_PARTIAL to the peer we received the data from. (Actually, maybe this shouldn't be done).
• PROPOSE_UPGRADE:
• After acting on a PROPOSE_UPGRADE, a restart is required before v3auth will be implemented. This can be resolved.
• Currently we PROPOSE_UPGRADE on every connection. We can avoid that if we store the current value of the connection and not just peerversion.
• There are a few places we can be more careful about not de-referencing null pointers. Search blake3.c and base32.c for " *" (aka \t*). The blake3 functions do no sanity checks on arguments passed, and the base32_decode function can be to be re-written (in stack).
• "Include the system error text (from perror or equivalent) in every error message resulting from a failing system call" https://www.gnu.org/prep/standards/html_node/Semantics.html
• Onloading/offloading old messages:
• Offloading: Should leave us with just -10 to 10 initialized, on all peers
• delete_log should probably be broken up into log_delete and shrink_message_struct, where log_delete calls shrink_message_struct, but shrink_message_struct can be separately called. Both could take n (or -1) as args.
• Cache Solution Proposal #1 (bad)
• When offloaded, we cache on the initialized message struct, then when re-loading, we just move to a temp local pointer, then sql_populate_message, then add back the cached messages via load_message_struc
• Cache Solution Proposal #2 (better)
• We offload all messages, except the 10 most recent, which we move to 0 to 10.
• If we keep the ~50 most recent instead of 10, we can just rely on load_more later.
• Remaining issue: We can't just offload all except a set number because then group_peer messages won't align with the appropriate group_n messages.
• Add a `message_spoof(n,time,nstime,data,len)` function where users can click between messages to add a fake message to their history. Primary Issue: How we fit this into the message struct? This is where a linked list struct would be handy. Secondary issue: Is there significant value? The order in which it is inserted into SQLCipher might give it away as non-genuine.
• Multiple offers of the same file with different filenames to the same peer has some weird behavior but is safe.
• Documentation: As of 2.0.10, PIPE_AUTH is signed and utilized when v3auth is not available (Tor <0.4.6.1) to facilitate secure full-duplex connections.
• Documentation: PROPOSE_UPGRADE works for upgrades from v1 (PIPE_AUTH) to v2 (v3auth), but cannot facilitate downgrades (and for many reasons should/will never). However, if both peers upgrade then disable v3auth, it will successfully fall back to PIPE_AUTH.
• Where multiple getter functions occur in a block, it would be more efficient to manually lock/unlock once.
• Consider implementation of Proof of Work https://spec.torproject.org/hspow-spec/v1-equix.html for multiple use IDs, when sufficiently mature
• Convert or eliminate remaining ~100 printf to error_printf
• Takedown_onion must call disconnect_forever in a threadsafe manner. We've tried everything (evbuffer_lock, event_base_once, etc) and can't find a way.
• file_init() calculates a checksum every time it offers a file. We have modification time now but we don't use it for anything.
• Seems like old file descriptors get closed. If nothing is written in a long time, fd might get closed. Do we have any long-term fd?
• File descriptor leaks and limitations: ulimit -n shows we can only have 1024 file descriptors open at once. That's going to limit the potential of group chats. If we get near the limit, we could shed outbound (sendfd) connections? Not a great solution.
• Lots of warnings occur when restarting Tor.
• Cascade deletion of messages and peer specific settings may be untested
• We don't have library protection against running twice. We implement "running" check in GTK. For now, this is ideal. No reason to change this.
• Is --keygen/--key-expiration only for relays? OfflineMasterKey/SigningKeyLifetime
• man atexit. May be useful. Note: Doesn't trigger on signals (ie ctrl + c). Not sure why this would be useful since we handle all exit() calls.
• Add a counter or bunch of counters that increment every time a malloc occurs, so we can know how many leaks we got. (optionally in what areas)
• Public Groups could hypothetically also be shortened in length (pointless), or allow external generation (pointless)
• Consider having UTF8 validator shift forward over any non-utf8 bytes, for use in libevent.c and perhaps message_send (to avoid sql issues), rather than discarding whole messages.
• For GROUP_PEER, .message and .message_len could be double pointer + pointer, respectively. Unsure if viable and how we would re-structure our message struct for that. This can be ignored for now so long as we aren't having bugs.
• Consider ripping out libsodium and replacing it with with openssl (since it is already a tor dependency) or libressl. Be sure that we can do everything in onion_gen.c (ex: crypto_sign_ed25519_pk_to_curve25519)
• Consider moving the message size and protocol behind the .message pointer (so we can avoid copying moving message when signing).
• Main issue is that then freeing messages would require a special torx_free, which is very bad. Not worth it.
• Kill codes have never been tested in groups because deleting individual peers in groups is a complex concept (since they'll just be re-added)