Indeed SQL is very similar to actor-critic method which has a soft Q-function critic network with parameter $\theta$ and an actor policy network with parameter $\phi$, and in fact the paper "Equivalence Between Policy Gradients and Soft Q-Learning" by Schulman et al proves equivalence between the gradient of soft Q-learning within maximum entropy RL framework and natural policies with entropy regularization in its gradient estimator, and further talked about soft Q-learning:
Haarnoja et al. [2017] work in the same setting of soft Q-learning as the current paper, and they are
concerned with tasks with high-dimensional action spaces, where we would like to learn stochastic
policies that are multi-modal, and we would like to use Q-functions for which there is no closed-form way of sampling from the Boltzmann distribution $π(a|s) ∝ π(a|s) exp(Q(s,a)/τ)$. Hence, they use a method called Stein Variational Gradient Descent to derive a procedure that jointly updates the Q-function and a policy $π$, which approximately samples from the Boltzmann distribution—this resembles variational inference, where one makes use of an approximate posterior distribution.
Having said that, in terms of Haarnoja et al's own words about the subtle difference between SQL and actor-critic method in your first reference below, the estimated sampled action from SVGD network is not used directly to affect the next soft Q-function since the critic network uses minibatch experiences from its stored replay memory to update its parameter instead of the usual advantage function. If it's not sampled accurately enough it may not be stable and converge to an optimal final (stochastic) solution.
Although the soft Q-learning algorithm proposed by Haarnoja et al. (2017) has a value function and actor network, it is not a true actor-critic algorithm: the Q-function is estimating the optimal Q-function, and the actor does not directly affect the Q-function except through the data distribution. Hence, Haarnoja et al. (2017) motivates the actor network as an approximate sampler, rather than the actor in an actor-critic algorithm. Crucially, the convergence of this method hinges on how well this sampler approximates the true posterior.
