====== Hybris' persistence mechanism ===== When Hybris was first released in 1997, the first version Enterprise JavaBeans (EJB) was still immature and Hibernate ORM didn't exist. So Hybris developed its **own proprietary persistence mechanism which generates Java classes from an xml file**. These classes and their relations can easily stored into a relational database. As this is the only persistence mechanism supported by Hybris, every backend developer working with the e-commerce suite must understand how it works and its **limitations.** ====== Old Jalo classes ====== There was an evolution in the Hybris ORM. Until version Hybris 4, you could only generate Jalo classes which are **Java persistent classes containing business logic**. This led to spaghetti code on those classes. Because of this, Models which are Java persistent POJOs were introduced. Because it is a bad practice to use Jalo classes, I won't explain them here. ====== Defining persistent classes in items.xml ====== Note: I will use the source code of the Hybris extension [[https://sourceforge.net/p/arecodeploymentscriptsmanager/code/ci/master/tree/|Areco Deployment Scripts Manager]] to explain how the persistence mechanism works.\\ \\ Every Hybris extension contains the file **resources/-items.xml** which is used to define the persistent classes which are used by the extension. Let take a closer look to the file arecoDeploymentScriptsManager-items.xml Location of the deployment script Name of the deployment script. It is usually the directory name. Final state of the execution When was the deployment script run. Stacktrace if the script was unsuccessful. It is 4KB long. text varchar(4096) * Each persistent class is called **Type** in Hybris and each instance is an **Item**. The code is just the name of the type and it is usually the name of the Java class * Each Hybris type has a jaloclass. You shouldn't use it as it is deprecated but for compatibility reasons, it is required and generated by Hybris * For each Hybris type the following Java classes are generated: * **Model Classes** like org.areco.ecommerce.deploymentscripts.model.ScriptExecutionModel: **Models are persistent POJOs without any business logic**. They are saved in a jar file in the core extension and every extension can use them. They are usually attached to a persistence session. * Deprecated **Jalo Classes** like org.areco.ecommerce.deploymentscripts.jalo.ScriptExecution: They used to contain persistence logic plus business logic. The absence of separation of concerns led to spaghetti code * **DTO Classes** like org.areco.ecommerce.deploymentscripts.dto.ScriptExecutionDTO. These are POJOs used by the extension **platformwebservices** for the marshalling and unmarshalling of the responses and requests. Unlike the models, they can't contain any business logic and can't be attached to a persistence session. They are rarely used * **Resource Classes** like org.areco.ecommerce.deploymentscripts.resource.ScriptExecutionResource: This is a Java class which was used for marshalling and unmarshalling of collections of responses and requests by the extension **platformwebservices** * **Database Table** * Every Hybris type is a **subclass of the class Item** which uses the table genericitems * Hybris puts the items of the Hybris types by default in the table genericitems. This leads to performance issues when the number of Hybris types increases. [[https://help.hybris.com/6.5.0/hcd/8c6254f086691014b095a08a61d1efed.html|Due to this it is good practice to declare a deployment table for every direct subclass of Item]] * **if you forgot to add a deployment table, there is no way to change the table where the items are stored without doing a initialization of the database.** Adding the deployment table to the items.xml later on, don't have any effect * **Don't specify a second deployment table in your Hybris types because Hybris don't support it.** For example, you declare the Type EnergyProduct as subtype of Product with the deployment table energy_products and you have a Hybris type ProductCarouselCMSComponent with the attribute Product. Then you use Hybris' query language, FlexibleSeach, to look for carousels with products with stock, you will get duplicate carousels because Hybris does two joins for the tables products and energy_products with cmscomponents and aggregates them using **union all** * The subclasses of your custom Hybris items will all reside in the same database table * **Primary Key and type code** * The typecode is an internal ID for the Hybris type. It can't be changed once the type was created because **the primary keys of the items contain the typecode.** * The typecode ranges 0-10099, 13200-13299, 24400-24599 and 32700-32799 are reserved for internal Hybris' use and must not be used to avoid conflicts with new Hybris out-of-the-box extensions * Every item has the primary key (PK), creation time, modified time, itemtype (read-only) attributes which are mapped to columns in the table. They are managed by Hybris * **There is no way to generate valid PKs outside the Hybris platform and the PK of an item of one database could be invalid in other database.** Hybris has internal counters for PKs which aren't accessible to the developers. If you want to import data into another system, you must use Hybris export language Impex and regenerate the PK during the import. Due to this, **Hybris items have a secondary key** like uid for CMS components or "code and catalog version" for products * You can **extend Hybris' Types** like in the example with the attribut Product.isDangerousGood. You have to use autocreate="false" and the original jalo class * Database **indexes** can also be defined after the attribute definition ===== Recommendations ===== * **Database Tables:** Use the deployment tag to specify the table where the instances of this type or relation are going to be stored. A recommended practice is to do this with every type which is a direct subclass of Item or GenericItem and every relation. See [[https://wiki.hybris.com/display/release4/Specifying+a+Deployment+for+hybris+Platform+Types|https://wiki.hybris.com/display/release4/Specifying+a+Deployment+for+hybris+Platform+Types]] * If you specify a table for a Hybris Type which already has one, like Product, you will see duplicate row in HMC when ordering by the product attribute (Version 6.1). The sql create by the flexible search query service will join the tables incorrectly. * **Serializable:** Declare a //serialVersionUID%// field if the item could be used in a Collection. Hybris stores Collection in a blog field in the database after serializing the object. You may have [[http://www.javapractices.com/topic/TopicAction.do?Id=45|incompatibilities with future changes if you don't set your own serialVersionUID]]. See: * Don't change the package of an Hybris type or prepare yourself for unexpected errors during the update running system with the actual instances of the type. ====== Attributes ====== * They are the fields of your persistent Java classes and are going to be mapped to **columns on the deployment table** * The qualifier is the name of the field and is going to be used in the setter and getters names * The **type** can be an simple Java class which can be stored in a table column like Integer, String, Boolean (Atomic types) or a Hybris Type. In the later case, the PK of the Item is stored on the column * It is a good practice to always write a description for the attribute * You could use **modifiers** to tell Hybris if: * The field is **mandatory** (optional=false). The table column will allow nulls, the validation is done by Hybris using Java code * The field can only be written once (initial=true) * The field must be removed when the parent Item is removed (partof=true) Hybris don't trigger the remove interceptors when updating a partof relations. If a base product is updated with a **subset** of the variants which are on the database, **the missing variants will be removed** and you can't use a remove interceptor to prevent the deletion. This happens on SAP commerce 1905. \\ To really prevent the removal of products from the database you need a database trigger like the following one for Microsoft SQL Server: return jaloSqlScriptService.runDeleteOrUpdateStatement("CREATE OR ALTER TRIGGER PRODUCTS_DEL ON products INSTEAD OF DELETE AS THROW 51000, 'Products must not be deleted', 1") This code create database trigger preventing any removal of products. It depends on [[https://github.com/arobirosa/areco-deployment-script-manager/wiki|Areco Deployment Script Manager]] * The field is part of the secondary key and must be **unique** inside the cluster of servers (unique=true). A group of attributes could be unique. The validation is done by Hybris using Java code. As two threads may store two items with the same unique values on the database, you have to use database transactions to be sure that the fields are unique across the cluster of servers * [[https://help.hybris.com/6.5.0/hcd/8bff7a568669101488a5e40cb7bbd0b9.html#itemsxmlelementreference-type_modifierstype|Other modifiers]] * There are two used persistence types: * **Property**: The value of the field is stored into the database. The type of the column can be defined for each database engine like with the attribute stacktrace above * **Dynamic**: The setter and getter of the field call a Spring Bean which handles the storage of the value in any persistent medium. For example: (...) @Component("isDangerousGoodAttributeHandler") public class IsDangerousGoodAttributeHandler implements DynamicAttributeHandler { @Override public Boolean get(final ProductModel pProduct) { // If all the fields are empty, the product is not a dangerous good. return !(pProduct.getHazardStatementCodes().isEmpty() // && pProduct.getHazardStatementTexts().isEmpty() // && pProduct.getEuHazardStatementCodes().isEmpty() // && pProduct.getEuHazardStatementTexts().isEmpty() // && pProduct.getPrecautionaryStatementCodes().isEmpty() // && pProduct.getPrecautionaryStatementTexts().isEmpty() // && pProduct.getHazardPictographCodes().isEmpty() // && pProduct.getHazardPictographDescriptions().isEmpty() // && pProduct.getHazardSignalWords().isEmpty() // && pProduct.getBiocideNotes().isEmpty()); } @Override public void set(final ProductModel pProductModel, final Boolean pBoolean) { throw new UnsupportedOperationException("This attribute can't be set."); } } This handler makes a calculation based on other fields. Every handler must implement the interface **DynamicAttributeHandler** * Attributes can be **localized** and there would be one value for each locale. Internally Hybris uses a map: Description of the state. The values of these attributes are saved in a separated table with the sufix **lp** like arscriptresultlp for the type ScriptExecutionResult. ===== Recommendations ===== * Hybris doesn't remove old attributes when you remove them from the files *-items.xml. If you remove a mandatory attribute, you must set it as optional, remove the setters and getters: DEPRECATED: hybris doesn't remove old mandatory attributes, so we keep it. After an update running system in production, you could remove it. This means that it takes two releases to remove it.\\ The best alternative is to use the [[http://sourceforge.net/projects/arecodeploymentscriptsmanager/|Areco Deployment Script Manager]] and remove the old attribute using a deployment script. * Hybris can't guarantee the uniqueness of a group of attributes which have the modifier **unique**. As the uniqueness is check on the Java code side, you could end with duplicate rows in the database coming from different threads on one server or from different servers in a cluster. A good practice is to declare a unique index for the Hybris type or [[https://help.sap.com/viewer/d0224eca81e249cb821f2cdf45a82ace/1905/en-US/8c7387f186691014922080f2e053216a.html|use transactions]] for objects which must be consistent like stock. ====== Relations between Hybris Types ====== ===== One to many relations ===== This example creates the column **customer** in the table where the items EmailAddress are stored and getters and setters on the customers to set and get the list of emails. The sourceElement and targetElement may have modifiers like the attributes. You have to read the source and target elements XML tags crossed to understand where the setters and getters are generated. One to many relations cannot be localized. You have to use a many to many relationship and manage the removal of associations. ===== Many to many relations ===== This examples creates the **table voucherToInterestGroup with the links** between voucher restrictions and the interest groups. And generates the fields InterestGroupRestriction.interestGroups and InterestGroup.voucherRestrictions. A deployment table is required. ====== Hybris Enumerations ====== This static enumeration is generated as the Java enum org.areco.ecommerce.deploymentscripts.**enums**.SystemPhase and can be referenced in any attribute of Hybris types. **The list of values can only changed during compilation time by changing the items.xml file.**\\ If you want to change the values of an enumeration during **runtime**, creating values with hmc or importing them with Impex you have to declare an enumeration as dynamic: Flag for real stock level status This creates a Java class which has an static field for each code. This class has the field code and from the point of view of a developer, it is used like a Hybris type and you can import or export the items using Impex. If you need more fields in your custom dynamic enumeration, you must declare a Hybris type instead. ===== Recommendations ===== * **Hybris doesn't recommend using collections**, use relations instead * Hybris may remove support for CollectionTypes directly as the type of an attribute starting with Version 5.0. * They can't be searched. * They are saved as a list of PK in a blog column in the database. This is slow and you can use SQL or flexible Search to join tables by collection attributes. * Cases where you can use collections * When your field contains a collection of String or Enumeration values. * When your field is dynamic because you aren't persisting the collection in the database. * **Getter of a Collection**: **The returned Collection can't be modified** because you are going to get an Exception. [[https://help.sap.com/viewer/d0224eca81e249cb821f2cdf45a82ace/6.5.0.0/en-US/8c146aa686691014bbf6fe76a708ece2.html?q=Models%20collections|A new Java collection should be used]] * **Relations with enumerations:** 1-n don't work and m-n can be defined but you may get an error in some machines when running the update running system. A safe approach is to create an item type which relates the enumeration with the other item. * **One to Many**: Either endpoint cannot be copied easily because the information of the relation is kept in the instance of each class. * **Many to Many**: Can be copied and searched. The relations are kept in a separated table, so they can be searched with FlexibleQuery or SQL. * Don't create attributes in relations because they aren't supported by the model layer. If you need a relation with fields, create an association object —a new Hybris type—. ====== How to persist and load items ====== In every case you must use the **modelService**: @Component("deploymentScript2ExecutionConverter") public class DeploymentScript2ExecutionConverter implements Converter { private static final Logger LOG = Logger.getLogger(DeploymentScript2ExecutionConverter.class); @Autowired private ModelService modelService; /* * (non-Javadoc) * * @see de.hybris.platform.servicelayer.dto.converter.Converter#convert(java.lang.Object) */ @Override public ScriptExecutionModel convert(final DeploymentScript source) throws ConversionException { return this.convert(source, (ScriptExecutionModel) this.modelService.create(ScriptExecutionModel.class)); // Creation of a new item } /* * (non-Javadoc) * * @see de.hybris.platform.servicelayer.dto.converter.Converter#convert(java.lang.Object, java.lang.Object) */ @Override public ScriptExecutionModel convert(final DeploymentScript source, final ScriptExecutionModel execution) throws ConversionException { ServicesUtil.validateParameterNotNullStandardMessage("source", source); ServicesUtil.validateParameterNotNullStandardMessage("execution", execution); if (DeploymentScript2ExecutionConverter.LOG.isDebugEnabled()) { DeploymentScript2ExecutionConverter.LOG.debug("Creating an script execution model from the deployment script " + source); } execution.setExtensionName(source.getExtensionName()); // The model is filled execution.setScriptName(source.getName()); execution.setResult(null); // The caller must set the result before saving the execution. execution.setPhase(source.getPhase()); return execution; } } @Service public class ArecoDeploymentScriptsRunner implements DeploymentScriptRunner { private static final Logger LOG = Logger.getLogger(ArecoDeploymentScriptsRunner.class); @Autowired private ModelService modelService; (...) private void saveAndLogScriptExecution(final UpdatingSystemExtensionContext context, final ScriptExecutionModel scriptExecution) { this.modelService.save(scriptExecution); // The model is stored into the database context.logScriptExecutionResult(scriptExecution); } } To **retrieve** the model you may use ModelService#get(de.hybris.platform.core.PK) but you usually use Hybris Query Language **FlexibleSearch** to get items from the database.\\ You can **remove** an item using ModelService#remove(java.lang.Object).\\ You can **update the item with the contents of the database** using ModelService#refresh. If two thread modify simultaneously two instances of the same model containing different values, **the last thread is going to override the changes from the first one**. If you are updating critical models like stock levels you must do it inside a database transaction. Hybris don't have any optimistic locking mechanism like Hibernate ORM. ====== Localization of Hybris types and attributes ====== The names and descriptions of the Hybris types, enumerations, relations and attributes can be localized using a property file inside **resources/localization**. For example, File arecoDeploymentScriptsManager/resources/localization/arecoDeploymentScriptsManager-locales_de.properties (...) type.DeploymentEnvironment.name=Deployment Umgebungen type.DeploymentEnvironment.name.name=Name type.DeploymentEnvironment.description.name=Beschreibung type.SystemPhase.name=Phase des Systems type.SystemPhase.INITIALIZATION.name=Initialization type.SystemPhase.UPDATE.name=Aktualisierung (...) This sets the name of a Hybris type, an enumeration and two attributes in German. ====== General Recommendations ====== * Please read [[https://www.sap.com/cxworks/article/433893244/Data_Model_Design_with_the_SAP_Commerce_Cloud_Type_System|Hybris' general good practices on defining types.]] Those recommendations aren't on this page * Don't use the menu Backoffice > System > Types to modify the Hybris types on runtime because you changes will be lost after an update running system --Based on Hybris 6.5