Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add metadata fields to core entities (FE) #3212

Merged
merged 136 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
136 commits
Select commit Hold shift + click to select a range
f41c8d5
feat: Add metadata fields to core entities
novakzaballa Oct 13, 2023
e1bc9ec
feat: Add metadata model field
novakzaballa Oct 18, 2023
930c9c8
Add metadata to environments and segments
novakzaballa Oct 19, 2023
625e331
Solve error with metadataselect
novakzaballa Oct 19, 2023
2b0b9d5
Solve content types id
novakzaballa Oct 19, 2023
2293935
wrap the code with the enable_metadata flag
novakzaballa Oct 19, 2023
830a82a
Close MetadataSelect component
novakzaballa Oct 19, 2023
c1a0a7a
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Oct 20, 2023
95033ed
fix: Conflicts with merge
novakzaballa Oct 20, 2023
1a0c84a
fix: Cannot resolve useMetadata
novakzaballa Oct 20, 2023
db8f13a
fix: Cannot resolve useMetadata
novakzaballa Oct 20, 2023
a9c0490
fix: Rename useMetaData to useMetadata
novakzaballa Oct 20, 2023
2eac883
fix: Solve e2e tests
novakzaballa Oct 20, 2023
dc4d2a2
fix: Solve e2e test
novakzaballa Oct 20, 2023
74db03d
fix: metadata list styles
novakzaballa Oct 23, 2023
9f9b283
Fix behavior when metadata is Created or updated
novakzaballa Oct 24, 2023
5c65852
Update metadata UI
novakzaballa Oct 30, 2023
6f6512e
Update required metadata model fields for segments and flags
novakzaballa Oct 31, 2023
0b6c89b
Add test to metadata for features and segments
novakzaballa Nov 1, 2023
9a25d76
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Nov 1, 2023
8b08a3e
Update metadata UI required fields
novakzaballa Nov 3, 2023
557f68b
fix: Metadata for features
novakzaballa Nov 15, 2023
cc2bada
fix: Delete moved tests
novakzaballa Nov 15, 2023
c52cb3b
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Nov 17, 2023
14e0f92
fix responses
novakzaballa Nov 20, 2023
37e79e6
Delete disable, required, and enabled test, add a confirm modal to de…
novakzaballa Nov 20, 2023
5536f45
fix metadata search input
novakzaballa Nov 20, 2023
8a489a2
fix metadata search input
novakzaballa Nov 20, 2023
9ddf027
Update UpdateFeature serialier
novakzaballa Nov 20, 2023
aee6657
fix add metadata in features
novakzaballa Nov 21, 2023
2e19984
Add segment test and update segment serializer
novakzaballa Dec 14, 2023
3057e7c
Change UI in metadata tab
novakzaballa Dec 14, 2023
6f4871a
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Dec 14, 2023
a12ffa3
reformat files
novakzaballa Dec 14, 2023
429c673
reformat files
novakzaballa Dec 14, 2023
0275c09
Reformat file
novakzaballa Dec 14, 2023
9844cd5
Add description tooltip to metadata
novakzaballa Dec 15, 2023
8763b09
Add metadata docs
novakzaballa Dec 15, 2023
dc60ee4
Add metadata docs
novakzaballa Dec 15, 2023
28a5617
Adjust styles
novakzaballa Dec 19, 2023
0dc05e9
update test to support metadata query
novakzaballa Dec 21, 2023
1ec1125
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
matthewelwell Dec 21, 2023
d06f518
Add logs for debug test on cloud
novakzaballa Dec 21, 2023
6cbd20c
Remove debug logs and update test
novakzaballa Dec 21, 2023
3217c37
adjust metadata styles
novakzaballa Dec 22, 2023
cef4d4e
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Dec 22, 2023
264bb0d
Add new tests
novakzaballa Jan 9, 2024
df39f39
Use supported content types endpoint
novakzaballa Jan 9, 2024
0643314
Improve code
novakzaballa Jan 9, 2024
8bee370
Rename component and improve supported content types service
novakzaballa Jan 10, 2024
c743a6f
Rename component and improve supported content types service
novakzaballa Jan 10, 2024
b56f8bd
Rename component
novakzaballa Jan 10, 2024
5d0c952
Solve issues with the renamed file
novakzaballa Jan 10, 2024
f0bfc37
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Jan 12, 2024
f1a4bb0
solve errors
novakzaballa Jan 19, 2024
131e847
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Jan 19, 2024
0c4d817
Solve tests errors
novakzaballa Jan 23, 2024
3ab272b
Delete unnecessary code
novakzaballa Jan 23, 2024
b342d58
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 4, 2024
652a9d0
Remove API changes
novakzaballa Apr 4, 2024
67e474f
Fix npm docs error
novakzaballa Apr 4, 2024
e4fec07
Correct types, delete supportedContentTypes Request from store, use RTK
novakzaballa Apr 4, 2024
30af6f6
Delete comment
novakzaballa Apr 4, 2024
3a6864c
Correct segment types
novakzaballa Apr 4, 2024
3341574
Change types in CreateMetadata file
novakzaballa Apr 4, 2024
fba77d8
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 7, 2024
6282e28
Solve getContentType types
novakzaballa Apr 7, 2024
661ecea
Create MetadataPage component
novakzaballa Apr 7, 2024
2006f79
Refactor feature Metadata select
novakzaballa Apr 7, 2024
dfb7ea4
Change class names
novakzaballa Apr 8, 2024
c7f1f06
Create new metadata components
novakzaballa Apr 8, 2024
98efcd7
New create metadata local logic
novakzaballa Apr 9, 2024
e0036c4
New flow to create MetadataField and MetadataModelField
novakzaballa Apr 9, 2024
85da11c
Update and delete metadata
novakzaballa Apr 10, 2024
f224fdd
Changes variable names
novakzaballa Apr 10, 2024
18b30b5
Change table for a panel search
novakzaballa Apr 10, 2024
77b33ac
Delete unnecesary code from _list.css
novakzaballa Apr 10, 2024
1677ac5
Delete redundant function
novakzaballa Apr 10, 2024
8a87411
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 10, 2024
b44b863
Fix import error
novakzaballa Apr 10, 2024
7453a51
Add Metadata Docs
novakzaballa Apr 10, 2024
08d9797
change update
novakzaballa Apr 10, 2024
d87dff4
Clean logs
novakzaballa Apr 10, 2024
66ba1d6
Correct merge problem
novakzaballa Apr 10, 2024
a2b2f2f
Move button functionality to save function
novakzaballa Apr 10, 2024
25614d2
Use useMemo instead of useState
novakzaballa Apr 10, 2024
f8f7943
Solve create error
novakzaballa Apr 10, 2024
28872d6
Add content types to metadata table row
novakzaballa Apr 11, 2024
b87c99d
Rename type
novakzaballa Apr 11, 2024
71d5027
Rename type Metadata to MetadataField
novakzaballa Apr 11, 2024
f62bdfc
Create add metadata component
novakzaballa Apr 11, 2024
a2a95e1
Rename Metadata to MetadataField
novakzaballa Apr 11, 2024
2de5489
Improvements in the code
novakzaballa Apr 11, 2024
e985331
Improvement CreateMetadataField
novakzaballa Apr 11, 2024
2be4a6f
Update Documents, update UI in AddMetadataToEntity component
novakzaballa Apr 12, 2024
08318c4
UX/UI
novakzaballa Apr 12, 2024
3c70da6
Change UI metadata
novakzaballa Apr 15, 2024
c0b1f3f
UI/Ux
novakzaballa Apr 16, 2024
9e02761
Add UI changes
novakzaballa Apr 16, 2024
d32bba7
Update Environment metadata
novakzaballa Apr 16, 2024
1c9d52c
Solve error tests
novakzaballa Apr 16, 2024
e3ee0b5
delete duplicate code created by merge error
novakzaballa Apr 16, 2024
3e4392e
Solve error when try create a segment or a feature
novakzaballa Apr 17, 2024
ad2363d
deleted comment
novakzaballa Apr 18, 2024
02af18a
Delete Logs and comments
novakzaballa Apr 18, 2024
3e055c2
Move tooltips positions
novakzaballa Apr 18, 2024
2a70966
Add metadata to environment and segments
novakzaballa Apr 19, 2024
71ea34d
Add validation when the Metadata change
novakzaballa Apr 19, 2024
4be8797
Update types exports
novakzaballa Apr 19, 2024
6535df8
Add validation when save data
novakzaballa Apr 19, 2024
4824e33
Update create feature with metadata
novakzaballa Apr 22, 2024
ea652de
Update segments with metadata correction
novakzaballa Apr 22, 2024
011db90
Update feature-list-store
novakzaballa Apr 22, 2024
6f40645
Correct update environment with metadata
novakzaballa Apr 22, 2024
b395e7e
Delete unnecesary components
novakzaballa Apr 22, 2024
e883981
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 22, 2024
fc96a29
Solve format issues
novakzaballa Apr 22, 2024
ca4d00d
Clean code
novakzaballa Apr 23, 2024
086cf75
Solve docs format
novakzaballa Apr 23, 2024
0bcd0fe
Update Docs and UI
novakzaballa Apr 23, 2024
2d7c8ca
Solve format issues
novakzaballa Apr 23, 2024
29a11df
Delete logs
novakzaballa Apr 23, 2024
b56b012
Clean code
novakzaballa Apr 23, 2024
ac7cfa8
Add comments in mergeMetadataEntityWithMetadataField function
novakzaballa Apr 23, 2024
f971775
Clean code
novakzaballa Apr 23, 2024
fc965d2
Clean code
novakzaballa Apr 23, 2024
7145d96
Change examples, save onBlur, Change boolean with a switch
novakzaballa Apr 25, 2024
e2d0418
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 25, 2024
e90d205
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa Apr 30, 2024
8377d38
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa May 13, 2024
b145be7
docs: Move segments metadata docs to segments.md
novakzaballa May 13, 2024
61f2c5f
Solve scss issue
novakzaballa May 13, 2024
6e5f0ed
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa May 15, 2024
e46d285
Solve merge metadata error
novakzaballa May 15, 2024
5501e4b
Merge branch 'main' into feat/add-metadata-fields-to-core-entities
novakzaballa May 15, 2024
17124db
Disable button when feature has metadata required
novakzaballa May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions api/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,6 +505,46 @@ def required_a_environment_metadata_field(
return model_field


@pytest.fixture()
def required_a_feature_metadata_field(
organisation,
a_metadata_field,
feature_content_type,
project,
project_content_type,
):
model_field = MetadataModelField.objects.create(
field=a_metadata_field,
content_type=feature_content_type,
)

MetadataModelFieldRequirement.objects.create(
content_type=project_content_type, object_id=project.id, model_field=model_field
)

return model_field


@pytest.fixture()
def required_a_segment_metadata_field(
organisation,
a_metadata_field,
segment_content_type,
project,
project_content_type,
):
model_field = MetadataModelField.objects.create(
field=a_metadata_field,
content_type=segment_content_type,
)

MetadataModelFieldRequirement.objects.create(
content_type=project_content_type, object_id=project.id, model_field=model_field
)

return model_field


@pytest.fixture()
def optional_b_environment_metadata_field(organisation, b_metadata_field, environment):
environment_type = ContentType.objects.get_for_model(environment)
Expand Down Expand Up @@ -542,6 +582,16 @@ def environment_content_type():
return ContentType.objects.get_for_model(Environment)


@pytest.fixture()
def feature_content_type():
return ContentType.objects.get_for_model(Feature)


@pytest.fixture()
def segment_content_type():
return ContentType.objects.get_for_model(Segment)


@pytest.fixture()
def project_content_type():
return ContentType.objects.get_for_model(Project)
Expand Down
4 changes: 4 additions & 0 deletions api/features/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
SoftDeleteExportableModel,
abstract_base_auditable_model_factory,
)
from django.contrib.contenttypes.fields import GenericRelation
from django.core.exceptions import (
NON_FIELD_ERRORS,
ObjectDoesNotExist,
Expand Down Expand Up @@ -72,6 +73,7 @@
STRING,
)
from features.versioning.models import EnvironmentFeatureVersion
from metadata.models import Metadata
from projects.models import Project
from projects.tags.models import Tag

Expand Down Expand Up @@ -124,6 +126,8 @@ class Feature(

objects = FeatureManager()

metadata = GenericRelation(Metadata)

class Meta:
# Note: uniqueness index is added in explicit SQL in the migrations (See 0005, 0050)
# TODO: after upgrade to Django 4.0 use UniqueConstraint()
Expand Down
30 changes: 27 additions & 3 deletions api/features/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from environments.sdk.serializers_mixins import (
HideSensitiveFieldsSerializerMixin,
)
from metadata.serializers import MetadataSerializer, SerializerWithMetadata
from projects.models import Project
from users.serializers import (
UserIdsSerializer,
Expand Down Expand Up @@ -245,11 +246,34 @@ def get_num_identity_overrides(self, instance) -> typing.Optional[int]:
return None


class UpdateFeatureSerializer(ListCreateFeatureSerializer):
"""prevent users from changing certain values after creation"""
class FeatureSerializerWithMetadata(
SerializerWithMetadata, ListCreateFeatureSerializer
):
metadata = MetadataSerializer(required=False, many=True)

class Meta(ListCreateFeatureSerializer.Meta):
read_only_fields = ListCreateFeatureSerializer.Meta.read_only_fields + (
fields = ListCreateFeatureSerializer.Meta.fields + ("metadata",)

def get_project(self, validated_data: dict = None) -> Project:
view = self.context.get("view")

if view and "project_pk" in view.kwargs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if view and "project_pk" in view.kwargs:
if view and project_pk := view.kwargs.get("project_pk"):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected

project_pk = view.kwargs["project_pk"]
try:
return Project.objects.get(pk=project_pk)
except Project.DoesNotExist:
raise serializers.ValidationError("Project not found.")

raise serializers.ValidationError(
"Unable to retrieve project for metadata validation."
)


class UpdateFeatureSerializer(FeatureSerializerWithMetadata):
"""prevent users from changing certain values after creation"""

class Meta(FeatureSerializerWithMetadata.Meta):
read_only_fields = FeatureSerializerWithMetadata.Meta.read_only_fields + (
"default_enabled",
"initial_value",
"name",
Expand Down
7 changes: 4 additions & 3 deletions api/features/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
FeatureInfluxDataSerializer,
FeatureOwnerInputSerializer,
FeatureQuerySerializer,
FeatureSerializerWithMetadata,
FeatureStateSerializerBasic,
FeatureStateSerializerCreate,
FeatureStateSerializerFull,
Expand Down Expand Up @@ -103,9 +104,9 @@ class FeatureViewSet(viewsets.ModelViewSet):

def get_serializer_class(self):
return {
"list": ListCreateFeatureSerializer,
"retrieve": ListCreateFeatureSerializer,
"create": ListCreateFeatureSerializer,
"list": FeatureSerializerWithMetadata,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the metadata on the list endpoint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understanding how it was working in the environments, yes.

"retrieve": FeatureSerializerWithMetadata,
"create": FeatureSerializerWithMetadata,
"update": UpdateFeatureSerializer,
"partial_update": UpdateFeatureSerializer,
}.get(self.action, ProjectFeatureSerializer)
Expand Down
16 changes: 14 additions & 2 deletions api/metadata/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

FIELD_VALUE_MAX_LENGTH = 2000

METADATA_SUPPORTED_MODELS = ["environment"]
METADATA_SUPPORTED_MODELS = ["environment", "feature", "segment"]

# A map of model name to a function that takes the object id and returns the organisation_id
SUPPORTED_REQUIREMENTS_MAPPING = {
Expand All @@ -21,7 +21,19 @@
"project": lambda project_id: Project.objects.get(
id=project_id
).organisation_id,
}
},
"feature": {
"organisation": lambda org_id: org_id,
"project": lambda project_id: Project.objects.get(
id=project_id
).organisation_id,
},
"segment": {
"organisation": lambda org_id: org_id,
"project": lambda project_id: Project.objects.get(
id=project_id
).organisation_id,
},
}


Expand Down
4 changes: 4 additions & 0 deletions api/segments/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,15 @@
SoftDeleteExportableModel,
abstract_base_auditable_model_factory,
)
from django.contrib.contenttypes.fields import GenericRelation
from django.core.exceptions import ValidationError
from django.db import models
from flag_engine.segments import constants

from audit.constants import SEGMENT_CREATED_MESSAGE, SEGMENT_UPDATED_MESSAGE
from audit.related_object_type import RelatedObjectType
from features.models import Feature
from metadata.models import Metadata
from projects.models import Project

logger = logging.getLogger(__name__)
Expand All @@ -35,6 +37,8 @@ class Segment(
Feature, on_delete=models.CASCADE, related_name="segments", null=True
)

metadata = GenericRelation(Metadata)

class Meta:
ordering = ("id",) # explicit ordering to prevent pagination warnings

Expand Down
22 changes: 22 additions & 0 deletions api/segments/serializers.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
import typing

from django.contrib.contenttypes.models import ContentType
from flag_engine.segments.constants import PERCENTAGE_SPLIT
from rest_framework import serializers
from rest_framework.exceptions import ValidationError
from rest_framework.serializers import ListSerializer
from rest_framework_recursive.fields import RecursiveField

from metadata.models import Metadata
from metadata.serializers import MetadataSerializer
from projects.models import Project
from segments.models import Condition, Segment, SegmentRule

Expand Down Expand Up @@ -41,6 +44,7 @@ class Meta:

class SegmentSerializer(serializers.ModelSerializer):
rules = RuleSerializer(many=True)
metadata = MetadataSerializer(required=False, many=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm keen for this not to be included on any serializers that are used for list views, which I think this is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this was answered here: #3315


class Meta:
model = Segment
Expand All @@ -58,12 +62,14 @@ def create(self, validated_data):
self.validate_project_segment_limit(project)

rules_data = validated_data.pop("rules", [])
metadata_data = validated_data.pop("metadata", [])

# create segment with nested rules and conditions
segment = Segment.objects.create(**validated_data)
self._update_or_create_segment_rules(
rules_data, segment=segment, is_create=True
)
self._update_or_create_metadata(metadata_data, segment=segment)
return segment

def validate_project_segment_limit(self, project: Project) -> None:
Expand Down Expand Up @@ -120,6 +126,22 @@ def _update_or_create_segment_rules(
child_rules, rule=child_rule, is_create=is_create
)

def _update_or_create_metadata(self, metadata_data, segment=None):
for metadata_item in metadata_data:
metadata_id = metadata_item.pop("id", None)
if metadata_item.get("delete"):
Metadata.objects.filter(id=metadata_id).delete()
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused here - I don't think this is how it works for other entities?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this was answered here: #3315


Metadata.objects.update_or_create(
id=metadata_id,
defaults={
**metadata_item,
"content_type": ContentType.objects.get_for_model(Segment),
"object_id": segment.id,
},
)

@staticmethod
def _update_or_create_segment_rule(
rule_data: dict, segment: Segment = None, rule: SegmentRule = None
Expand Down
38 changes: 37 additions & 1 deletion api/tests/unit/features/test_unit_features_views.py
Original file line number Diff line number Diff line change
Expand Up @@ -2219,6 +2219,42 @@ def test_cannot_update_feature_of_a_feature_state(
)


@pytest.mark.parametrize(
"client",
[lazy_fixture("admin_master_api_key_client"), lazy_fixture("admin_client")],
)
def test_create_feature_with_required_metadata_returns_201(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about other tests? e.g. trying to create a feature without the required metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

project,
client,
required_a_feature_metadata_field,
):
# Given
url = reverse("api-v1:projects:project-features-list", args=[project.id])
description = "This is the description"
field_value = 10
data = {
"name": "Test feature",
"description": description,
"metadata": [
{
"model_field": required_a_feature_metadata_field.id,
"field_value": field_value,
},
],
}

# When
response = client.post(url, data=json.dumps(data), content_type="application/json")

# Then
assert response.status_code == status.HTTP_201_CREATED
assert (
response.json()["metadata"][0]["model_field"]
== required_a_feature_metadata_field.field.id
)
assert response.json()["metadata"][0]["field_value"] == str(field_value)


def test_create_segment_override__using_simple_feature_state_viewset__allows_manage_segment_overrides(
staff_client: APIClient,
with_environment_permissions: Callable[
Expand Down Expand Up @@ -2399,7 +2435,7 @@ def test_list_features_n_plus_1(
v1_feature_state.clone(env=environment, version=i, live_from=timezone.now())

# When
with django_assert_num_queries(14):
with django_assert_num_queries(15):
response = staff_client.get(url)

# Then
Expand Down
Loading