Yes, you can delete commands from the history. They added a TUI command inspector, which gives you more information on commands (date/time of last run, a time series of command runs, exit codes, etc.). You can delete history entries from this inspector.

In terms of disabling the up arrow remap, it’s just a flag in the configuration file (detailed for each distro in the installation instructions).

I haven’t used atuin a lot yet, but I sure like having a fast, full screen command history with fuzzy search.