Permission-related unit tests are not isolated

[khvn26]

How are you running Flagsmith

• Self Hosted with Docker
• Self Hosted with Kubernetes
• SaaS at flagsmith.com
• Some other way (add details in description below)

Describe the bug

When testmon collects only a subset of permission-related tests, they fail due to empty table:

FAILED tests/unit/environments/identities/traits/test_traits_views.py::test_user_with_manage_identities_permission_can_add_trait_for_identity - permissions.models.PermissionModel.DoesNotExist: PermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::ProjectTestCase::test_create_project_returns_403_if_user_is_not_organisation_admin - organisations.permissions.models.OrganisationPermissionModel.DoesNotExist: OrganisationPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::ProjectTestCase::test_user_with_create_project_permission_can_create_project - organisations.permissions.models.OrganisationPermissionModel.DoesNotExist: OrganisationPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::ProjectTestCase::test_user_with_create_project_permission_cannot_create_project_if_restricted_to_admin - organisations.permissions.models.OrganisationPermissionModel.DoesNotExist: OrganisationPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::ProjectTestCase::test_user_with_view_project_permission_can_get_their_permissions_for_a_project - organisations.permissions.models.OrganisationPermissionModel.DoesNotExist: OrganisationPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::ProjectTestCase::test_user_with_view_project_permission_can_view_project - organisations.permissions.models.OrganisationPermissionModel.DoesNotExist: OrganisationPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserProjectPermissionsViewSetTestCase::test_user_can_create_new_user_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserProjectPermissionsViewSetTestCase::test_user_can_delete_user_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserProjectPermissionsViewSetTestCase::test_user_can_list_all_user_permissions_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserProjectPermissionsViewSetTestCase::test_user_can_update_user_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserPermissionGroupProjectPermissionsViewSetTestCase::test_user_can_create_new_user_group_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserPermissionGroupProjectPermissionsViewSetTestCase::test_user_can_delete_user_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserPermissionGroupProjectPermissionsViewSetTestCase::test_user_can_list_all_user_group_permissions_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::UserPermissionGroupProjectPermissionsViewSetTestCase::test_user_can_update_user_group_permission_for_a_project - projects.models.ProjectPermissionModel.DoesNotExist: ProjectPermissionModel matching query does not exist.
FAILED tests/unit/projects/test_unit_projects_views.py::test_can_list_project_permission[admin_client] - assert 0 == 6
 +  where 0 = len([])
 +    where [] = functools.partial(<bound method ClientMixin._parse_json of <rest_framework.test.APIClient object at 0x7f47059a83d0>>, <Response status_code=200, "application/json">)()
 +      where functools.partial(<bound method ClientMixin._parse_json of <rest_framework.test.APIClient object at 0x7f47059a83d0>>, <Response status_code=200, "application/json">) = <Response status_code=200, "application/json">.json
FAILED tests/unit/projects/test_unit_projects_views.py::test_can_list_project_permission[admin_master_api_key_client] - assert 0 == 6
 +  where 0 = len([])
 +    where [] = functools.partial(<bound method ClientMixin._parse_json of <rest_framework.test.APIClient object at 0x7f4704f591b0>>, <Response status_code=200, "application/json">)()
 +      where functools.partial(<bound method ClientMixin._parse_json of <rest_framework.test.APIClient object at 0x7f4704f591b0>>, <Response status_code=200, "application/json">) = <Response status_code=200, "application/json">.json
FAILED tests/unit/features/test_unit_features_views.py::ProjectFeatureTestCase::test_regular_user_can_update_feature_description - assert 404 == 200
 +  where 404 = <Response status_code=404, "application/json">.status_code
 +  and   200 = status.HTTP_200_OK
ERROR tests/unit/features/test_unit_features_views.py::test_create_segment_override__using_simple_feature_state_viewset__denies_update_feature_state - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(UPDATE_FEATURE_STATE) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_update_segment_override__using_simple_feature_state_viewset__denies_update_feature_state - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(UPDATE_FEATURE_STATE) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_list_features_group_owners - django.db.utils.IntegrityError: insert or update on table "projects_userprojectpermission_permissions" violates foreign key constraint "projects_userproject_permissionmodel_id_b332d105_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(VIEW_PROJECT) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_list_features_n_plus_1 - django.db.utils.IntegrityError: insert or update on table "projects_userprojectpermission_permissions" violates foreign key constraint "projects_userproject_permissionmodel_id_b332d105_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(VIEW_PROJECT) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_list_features_with_intersection_tag - django.db.utils.IntegrityError: insert or update on table "projects_userprojectpermission_permissions" violates foreign key constraint "projects_userproject_permissionmodel_id_b332d105_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(VIEW_PROJECT) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_list_features_with_union_tag - django.db.utils.IntegrityError: insert or update on table "projects_userprojectpermission_permissions" violates foreign key constraint "projects_userproject_permissionmodel_id_b332d105_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(VIEW_PROJECT) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_create_segment_override__using_simple_feature_state_viewset__allows_manage_segment_overrides - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(MANAGE_SEGMENT_OVERRIDES) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_simple_feature_state_returns_only_latest_versions - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(VIEW_ENVIRONMENT) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_update_segment_override__using_simple_feature_state_viewset__allows_manage_segment_overrides - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(MANAGE_SEGMENT_OVERRIDES) is not present in table "permissions_permissionmodel".
ERROR tests/unit/features/test_unit_features_views.py::test_create_segment_override_staff - django.db.utils.IntegrityError: insert or update on table "environments_userenvironmentpermission_permissions" violates foreign key constraint "environments_userenv_permissionmodel_id_5e08fd7a_fk_permissio"
DETAIL:  Key (permissionmodel_id)=(MANAGE_SEGMENT_OVERRIDES) is not present in table "permissions_permissionmodel".
ERROR tests/unit/environments/identities/test_unit_identities_views.py::test_user_with_view_environment_permission_can_not_list_identities - permissions.models.PermissionModel.DoesNotExist: PermissionModel matching query does not exist.
ERROR tests/unit/environments/identities/test_unit_identities_views.py::test_user_with_view_identities_permission_can_retrieve_identity - permissions.models.PermissionModel.DoesNotExist: PermissionModel matching query does not exist.
ERROR tests/unit/edge_api/identities/test_edge_identity_featurestate_view.py::test_user_with_view_environment_permission_can_retrieve_all_feature_states_for_identity - permissions.models.PermissionModel.DoesNotExist: PermissionModel matching query does not exist.

Example test run: https://github.com/Flagsmith/flagsmith/actions/runs/7732742740/job/21083333350

Steps To Reproduce

1. Change code in environments and projects apps.
2. File a pull request.

Expected behavior

Tests are properly isolated and run consistently regardless of test collection.

Screenshots

No response

[zachaysan]

To solve this we removed testmon which didn't compete with xdist in speed anyway.