alacritty is a terminal emulator. It's the drawing on your GUI that lets you do thing as if you were on the console.
tmux is a terminal multiplexer like screen that lets you run multiple terminal sessions within a single terminal but that can remain active even if you end your terminal application or connection.
You can connect to tmux from within alacritty or any other terminal emulator or ssh, etc connection. You can even connect to the same session from multiple terminals.
Start alacritty however you start it.
Type tmux attach || tmux new to attach to an existing session or start a new one.
Use tmux attach -t target-name || tmux new -s target-name to attach or create to a specific session.
Alternatively a script file:
~/bin/tmuxed:
#!/bin/bash
tmux attach -t MyFavSession || tmux new -s MyFavSession
Create a shortcut or edit the Alacritty menu entry to...
alacritty -e $HOME/bin/tmuxed
...to start alacritty with the tmux attach/new command automatically.