Repository navigation
Django: invalid VectorField input raises ValueError / DataError instead of a validation error #171
Description
Hi @ankane, thanks for pgvector-python!
When a VectorField gets malformed input through a ModelForm (including the admin) or a DRF ModelSerializer, the error isn't a validation error. With a non-numeric element, form.is_valid() raises ValueError. With the wrong number of dimensions, the form or serializer is valid and save() raises DataError from Postgres. Either way the request ends in a 500 instead of a form error or a 400 response.
Django's custom fields docs say to_python() should raise ValidationError when it can't convert a value. VectorField.to_python passes the string to Vector._from_text, and float() raises ValueError. For comparison, with ArrayField(FloatField()) the same kind of input gives a ValidationError: the model field rejects '["1", "x"]', and its form field rejects '1,2,x' with "Item 3 in the array did not validate".
HalfVectorField and SparseVectorField raise ValueError for malformed strings in the same way.
Reproduction
import django
from django.conf import settings
settings.configure(
DATABASES={'default': {'ENGINE': 'django.db.backends.postgresql', 'NAME': 'pgvector_python_test'}},
INSTALLED_APPS=[],
)
django.setup()
from django import forms
from django.db import connection, models
from pgvector.django import VectorField
from rest_framework import serializers
class Item(models.Model):
embedding = VectorField(dimensions=3)
class Meta:
app_label = 'repro'
with connection.cursor() as cursor:
cursor.execute('CREATE EXTENSION IF NOT EXISTS vector')
cursor.execute('DROP TABLE IF EXISTS repro_item')
with connection.schema_editor() as editor:
editor.create_model(Item)
class ItemForm(forms.ModelForm):
class Meta:
model = Item
fields = ['embedding']
class ItemSerializer(serializers.ModelSerializer):
class Meta:
model = Item
fields = ['embedding']
def check(name, fn):
try:
print(f'{name}: {fn()}')
except Exception as e:
print(f'{name}: {type(e).__name__}: {e}')
def form(value):
f = ItemForm(data={'embedding': value})
if not f.is_valid():
return f'invalid {f.errors.get_json_data()}'
f.save()
return 'saved'
def serializer(value):
s = ItemSerializer(data={'embedding': value})
if not s.is_valid():
return f'invalid {s.errors}'
s.save()
return 'saved'
check('form [1,2,x]', lambda: form('[1,2,x]'))
check('form [1,2]', lambda: form('[1,2]'))
check('serializer [1, 2, "x"]', lambda: serializer([1, 2, 'x']))
check('serializer [1, 2]', lambda: serializer([1, 2]))Output:
form [1,2,x]: ValueError: could not convert string to float: 'x'
form [1,2]: DataError: expected 3 dimensions, not 2
serializer [1, 2, "x"]: ValueError: could not convert string to float: 'x'
serializer [1, 2]: DataError: expected 3 dimensions, not 2
Traceback for the first case:
.../django/forms/models.py", line 503, in _post_clean
.../django/db/models/base.py", line 1713, in full_clean
.../django/db/models/base.py", line 1768, in clean_fields
.../django/db/models/fields/__init__.py", line 875, in clean
.../pgvector/django/vector.py", line 33, in to_python
.../pgvector/vector.py", line 112, in _from_db
.../pgvector/vector.py", line 88, in _from_text
ValueError: could not convert string to float: 'x'
Expected
All four cases come back as invalid with a field error, and nothing reaches the database. For the dimension mismatch, the field already knows dimensions, so it could check the length before saving.
Versions
- pgvector-python 0.5.1 (also on
master, e0f3ac3) - Django 6.1.2, Django REST framework 3.18.3, Psycopg 3.3.6
- Postgres 18.6, pgvector 0.8.7
- Python 3.12.7
Activity