Repository navigation
fix(join): report a join on unrelated namesakes, not missing lineage - #1642
dimitri-yatsenko wants to merge 1 commit into
Conversation
A join or restriction on a namesake attribute that has no lineage on one side was refused with a message about a stale `~lineage` entry, which told the user to run `schema.rebuild_lineage()`. But lineage comes from foreign keys: an attribute that no foreign key brings in has none by design. That includes a native secondary attribute, an attribute computed in a projection, and a key promoted by `dj.U`. Joining `Segmentation * Profile`, two tables that each declare a secondary `input_hash`, on a freshly declared schema raised the stale-lineage message, and rebuilding lineage could not change it. Such a join is now refused as a join on unrelated attributes. The message names the attribute, says which side does not inherit it through a foreign key, and suggests renaming or excluding it on one side with `.proj()`. The message for two different lineages is unchanged. Namesakes are checked in name order, so the same join always reports the same attribute. The new tests cover namesake secondary attributes of two tables (also after rebuilding lineage, on MySQL), computed attributes of two projections, keys promoted by `dj.U` on both sides, a native key against a secondary namesake, and the renaming and exclusion the message suggests. The test that expected the stale-lineage message now expects the new one. They fail before this change. Closes #1640.
MilagrosMarin
left a comment
There was a problem hiding this comment.
LGTM overall. Separating "no foreign key brings this attribute in" from "stale lineage" is the right call for the #1640 case, the join behavior itself is unchanged, and checking namesakes in sorted order makes the error deterministic. The new tests cover each way an attribute can lack lineage, plus the suggested remedies.
One suggestion on the one-sided message: stale lineage still happens, and the updated test_missing_lineage_is_an_unrelated_namesake shows it. It deletes Enrollment's lineage rows, so student_id, which does come from a foreign key, has no lineage, and the error now says the other side "does not inherit it through a foreign key". For a pre-2.3 schema that was never redecorated, that's confidently wrong, and the user loses the one hint that would fix it. Keeping a short clause in the one-sided case, e.g. "If it does come from a foreign key, its lineage may be stale: run schema.rebuild_lineage().", would cover both readings. The both-sides-None message can stay as it is.
A join or restriction on a namesake attribute that has no lineage on one side was refused with a message about a stale
~lineageentry, which told the user to runschema.rebuild_lineage(). But lineage comes from foreign keys: an attribute that no foreign key brings in has none by design. That includes a native secondary attribute, an attribute computed in a projection, and a key promoted bydj.U. JoiningSegmentation * Profile, two tables that each declare a secondaryinput_hash, on a freshly declared schema raised the stale-lineage message, and rebuilding lineage could not change it.Such a join is now refused as a join on unrelated attributes. The message names the attribute, says which side does not inherit it through a foreign key, and suggests renaming or excluding it on one side with
.proj(). The message for two different lineages is unchanged. Namesakes are checked in name order, so the same join always reports the same attribute.The new tests cover namesake secondary attributes of two tables (also after rebuilding lineage, on MySQL), computed attributes of two projections, keys promoted by
dj.Uon both sides, a native key against a secondary namesake, and the renaming and exclusion the message suggests. The test that expected the stale-lineage message now expects the new one. They fail before this change.Closes #1640.