Below is a summary (with precise references) of what is presently known when one removes the words “semisimple” and even “fusion’’ from the usual theory of module-categories.

---

1. Module categories (= actegories) in complete generality

---

The notion of a left, right or bi-module category over a monoidal category 𝓒 goes back to Bénabou, Kelly and Street:

• J. Bénabou, “Introduction to bicategories”, 1967.
• G. M. Kelly, M. L. Laplaza, “Coherence for compact closed categories”, 1980.
• R. Street, “Frobenius monads and pseudomonoids”, 2004 (= “modules over pseudomonoids”).

In that language a monoidal category is a pseudomonoid in the monoidal 2-category Cat, and a (bi)module category is simply a (bi)module for such a pseudomonoid. Everything in §0 of Ostrik works verbatim without linearity or rigidity.

---

2. Relative (or Deligne) tensor product of bimodule categories

---

For a right 𝓒-module 𝓜 and a left 𝓒-module 𝓝 one defines the tensor product

  𝓜 ⊠_𝓒 𝓝 := coeq ( 𝓜 ⊠ 𝓒 ⊠ 𝓝 ⇉ 𝓜 ⊠ 𝓝 )

characterised by the usual universal property for 𝓒-balanced functors.
Existence of that coequaliser is automatic if Cat is replaced by any bicategory having reflexive coequalisers that are preserved by the tensor product; for linear examples it is enough that the categories be finite abelian. Systematic treatments:

• B. Day & R. Street, “Quantum categories, star–autonomy and quantum groupoids”, QA/0406094, §7.
• P. Schauenburg, “Bimodules and 2-categories”, TAC 20 (2008), 581–605.
• A. Davydov, “Centre of an algebra”, Adv. Math. 225 (2010), 319–348, §7.

Hence for any monoidal 𝓒 satisfying those mild completeness hypotheses the 2-category   Bimod(𝓒) (bimodule categories, module functors, nat. transf.) is monoidal under ⊠_𝓒, exactly as you anticipated.

---

3. Finite tensor categories: replacing “fusion’’ by “finite’’

---

Take now a finite tensor category in the sense of Etingof–Ostrik (abelian, rigid, k-linear, finite length, End(1)=k); semisimplicity is dropped.

3.1 Existence of internal Homs
For 𝓒 finite, internal Homs 𝑼𝑛𝑑𝒆𝑟𝒍𝒊𝒏𝑒{Hom}(M,N) ∈ 𝓒 exist for every 𝓒-module category 𝓜 (EO, Lemma 2.1).

3.2 Classification of module categories
Etingof & Ostrik, “Finite tensor categories”, Mosc. Math. J. 4 (2004), Thm 3.30:
Every exact^1 indecomposable left 𝓒-module category 𝓜 is equivalent to the category 𝓒_A of right modules over an algebra object A := 𝑼𝑛𝑑𝑒𝑟𝒍𝒊𝒏𝑒{End}(m) for any generator m ∈ 𝓜.
(^1 Exact means the action of 𝓒 takes projectives to projectives; for semisimple 𝓒 this condition is automatic.)

Thus the basic result of Ostrik’s 2003 paper survives unchanged once semisimplicity is replaced by exactness.

3.3 Tensor product of module categories
If 𝓜 ≃ 𝓒_A and 𝓝 ≃ 𝓒_B, then (EO, Prop. 3.34)

  𝓜 ⊠_𝓒 𝓝 ≃ (𝓒_A) ⊠𝓒 (𝓒_B) ≃ 𝓒{A⊗B}

with its obvious A–B bimodule structure. Hence ⊠_𝓒 is again “given by tensoring the underlying algebras”. In particular the 2-category 𝓒-Mod_ex of finite exact module categories is monoidal.

---

4. Invertibility and Azumaya algebras

---

Definition (Davydov). An algebra object A in a braided finite tensor category 𝓒 is Azumaya iff the two canonical functors

  A ⊗ – : 𝓒 → 𝓒_A^A and – ⊗ A : 𝓒 → _A𝓒^A

are monoidal equivalences (or, equivalently, the monoidal functor 𝓒 → End_𝓒(𝓒_A) is an equivalence).

Main theorem (Davydov, Adv. Math. 225 (2010) 319–348, Thm 5.4).
For a braided finite tensor category 𝓒 the following are equivalent:

(a) 𝓒_A is an invertible 𝓒-bimodule category;
(b) A is Azumaya;
(c) The full centre Z(A) is trivial.

Consequently, the groupoid of Azumaya algebras in 𝓒 is equivalent to the groupoid of invertible 𝓒-bimodule categories, and any invertible bimodule comes from an Azumaya algebra.

---

5. The Brauer–Picard 3-group

---

For any finite tensor category 𝓒 the 3-group^2

  BrPic(𝓒) := inv.(𝓒-bimod. cat., module functors, nat. iso’s)

is defined in Etingof-Nikshych-Ostrik, “Fusion categories and homotopy theory”, 2009.

Thm 6.1 of that paper combined with Davydov’s result shows:

 BrPic(𝓒) ≃ Azumaya_3(𝓒)

where the right hand side is the 3-group whose 0-cells are Azumaya algebras, 1-cells invertible bimodules, 2-cells invertible bimodule morphisms. This is exactly the equivalence you asked about; semisimplicity is not needed—finite-ness and rigidity are enough.

(^2 “3-group’’ = monoidal 2-groupoid, or equivalently a bicategory all of whose k-morphisms are invertible.)

---

6. Remarks on still greater generality

---

• If rigidity is dropped one loses internal Homs, hence the proof of 𝓜≃𝓒_A breaks down; counter-examples exist.
• If 𝓒 is not finite abelian, ⊠_𝓒 may fail to exist. One can formulate everything inside any monoidal bicategory that has reflexive coequalisers preserved by ⊗, but few concrete classification theorems are known in that setting.

---

References (abridged)

A. Davydov, “Centre of an algebra”, Adv. Math. 225 (2010) 319–348.
P. Etingof, D. Nikshych, V. Ostrik, “Fusion categories and homotopy theory”, 2009.
P. Etingof, V. Ostrik, “Finite tensor categories”, Mosc. Math. J. 4 (2004).
B. Day, R. Street, “Quantum categories, star-autonomy and quantum groupoids”, 2004.
P. Schauenburg, “Bimodules and 2-categories”, TAC 20 (2008), 581–605.
R. Street, “Frobenius monads and pseudomonoids”, TAC 30 (2015) (original 2004 notes).

---

Answer to the concrete questions

1. “Is the module category of modules over an Azumaya algebra invertible?”
    Yes, for any braided finite tensor category (Davydov, loc. cit.).

2. “What is the tensor product of two such module categories?”
    If 𝓜 ≃ 𝓒_A and 𝓝 ≃ 𝓒_B then 𝓜 ⊠𝓒 𝓝 ≃ 𝓒{A⊗B}.

3. “Is there equality of the two Brauer 3-groups?”
    Yes, for every finite (not necessarily semisimple) braided tensor category 𝓒 one has
     BrPic(𝓒) ≃ Azumaya_3(𝓒).

So essentially every statement that was known for fusion categories continues to hold—as soon as one works with finite (k-linear, rigid) tensor categories and exact module categories. Beyond that level of generality very little is known at present.