boltzmann-gpt-beauty

The Beauty checkpoint of Energy-Based Attribute Models for Controllable Review Generation with Frozen LLMs (Junichiro Niimi, TMLR 2026). A Deep Boltzmann Machine over binary review attributes, plus the MLP adapter that turns its mean-field beliefs into soft prompts for a frozen Qwen/Qwen2.5-0.5B-Instruct. Load it with the boltzmann-gpt package.

• Paper: OpenReview
• Code: https://github.com/jniimi/boltzmann-gpt

Everything this model generates is synthetic review text. It is not a real customer's opinion and it describes no real purchase. Do not post generations as genuine reviews; if you publish or redistribute them, state clearly that they are model output.

Usage

uv add git+https://github.com/jniimi/boltzmann-gpt

from boltzmann_gpt import AttributeModel

model = AttributeModel.from_pretrained("jniimi/boltzmann-gpt-beauty")
model.attributes()                         # group -> allowed values

v = model.encode(brand="neutrogena", rating="5", topic=["Skincare"])
print(model.energy(v))                     # mean-field energy score, lower = more coherent
print(model.energy(model.clamp(v, topic=["Skincare", "Allergy"])))
print(model.generate(v, seed=0))           # renders via the frozen generator
print(model.generate(model.clamp(v, rating="1"), seed=0))

generate() downloads Qwen/Qwen2.5-0.5B-Instruct from the Hub on first use. Without a prompt= argument it uses the paper's one-shot prompt layout (paper, Appendix B): an instruction naming the domain ("beauty products"), one example review, and the task fields. The example review is synthetic and author-written, and the product is generic ("Moisturizing Face Cream", $19.99); both live in config.json. Override them with generate(v, product_name=..., price=..., example={"product_name": ..., "price": ..., "review": ...}), where a partial example dict is merged over the default. The average rating is 3.0 for both example and product, because it was constant at 3.0 in every training prompt. Pass prompt= to supply the full text prompt yourself; model.default_prompt(...) returns the prompt generate() would use.

What is in this repository

File	Contents
dbm.safetensors	DBM weights and biases, layers [170, 512, 1024, 2048]
adapter.safetensors	adapter MLP 3584 → 2048 → 4096 → 30×896 (ReLU) and the LayerNorm over the soft-prompt embeddings
config.json	architecture, mean-field iterations (10), generator id, default prompt
feature_spec.json	the 170 visible units: column names, attribute groups, modal defaults

The adapter reads the concatenation of all converged hidden-layer means (3584 dimensions) and emits 30 soft-prompt embeddings, which are prepended to the text prompt. The generator's weights are never modified. This is the seed-0 run reported in the paper; the other seeds are not released.

Attribute schema

Group	Type	Units	Values	Default (modal)
price	one-hot	4	Budget, Mid, Premium, Luxury	Mid
brand	one-hot	32	aveeno, avon, bath_body_works, cetaphil, clinique, colgate, covergirl, crest, dove, essie, estee_lauder, garnier, head_shoulders, herbal_essences, loreal, mac, mary_kay, maybelline, neutrogena, nivea, nyx, olay, opi, oral_b, others, pantene, revlon, sally_hansen, tarte, too_faced, tresemme, victorias_secret	others
rating	one-hot	5	1, 2, 3, 4, 5	5
topic	multi-label	17	Skincare, Haircare, Makeup, Fragrance, Nails, Color, Texture, Allergy, Lasting, Natural, AntiAging, SunCare, Price, Service, Gift, Quality, Packaging	(none)
purchase_flags	multi-label	5	Renewed, Over10Times, MaxOver500USD, MaxOver1000USD, MinOver100USD	(none)
purchase_freq	one-hot	3	Single, Few, High	Single
price_range	one-hot	4	Under10, 10to50, 50to100, Over100	Under10
auto_terms	multi-label	100	100 TF-IDF terms (see feature_spec.json)	(none)

Groups left out of encode() take their modal training value. The modal defaults are marginal modes taken group by group, so the default configuration as a whole is not a typical review (the topic and TF-IDF groups are empty, for instance) and its energy is higher than that of a typical training vector. For meaningful comparisons, start from a fully specified configuration.

Training data

Built from Amazon Reviews 2023, the All Beauty category: verified purchases, English only, one review per user; 52,952 / 1,024 / 1,024 train / validation / test reviews. Attribute construction is described in the paper's preprocessing appendix. No review text or user data is included here.

Limitations

The DBM models how attributes co-occur in this one training domain. It is not a causal model: clamping an attribute fixes visible units in the learnt distribution and re-runs mean-field inference, so the resulting changes reflect model-internal consistency, not real-world effects. Generations inherit the biases of the review corpus and of the generator, and a small generator often produces repetitive or ungrammatical text.

License

The DBM and adapter weights in this repository are released under the MIT License. The frozen generator, Qwen/Qwen2.5-0.5B-Instruct, is not included and is distributed by its authors under the Apache License 2.0.

Citation

@article{niimi2026energybased,
    title = {Energy-Based Attribute Models for Controllable Review Generation with Frozen {LLM}s},
    author = {Junichiro Niimi},
    journal = {Transactions on Machine Learning Research},
    issn = {2835-8856},
    year = {2026},
    url = {https://openreview.net/forum?id=pOIFHY4dOJ}
}