marshmallow
marshmallow是一個用來將復雜的orm對象與python原生數據類型之間相互轉換的庫,簡而言之,就是實現object -> dict
, objects -> list
, string -> dict
和 string -> list
。
要用到marshmallow,首先需要一個用於序列化和反序列化的類:
import datetime as dt
class User(object): def __init__(self, name, email): self.name = name self.email = email self.created_at = dt.datetime.now() def __repr__(self): return '<User(name={self.name!r})>'.format(self=self)
Schema
要對一個類或者一個json數據實現相互轉換(即序列化和反序列化,序列化的意思是將數據轉化為可存儲或可傳輸的數據類型),需要一個中間載體,這個載體就是Schema。除了轉換以外,Schema還可以用來做數據校驗。每個需要轉換的類,都需要一個對應的Schema:
from marshmallow import Schema, fields class UserSchema(Schema): name = fields.Str() email = fields.Email() created_at = fields.DateTime()
Serializing(序列化)
序列化使用schema中的dump()
或dumps()
方法,其中,dump()
方法實現obj -> dict
,dumps()
方法實現 obj -> string
,由於Flask能直接序列化dict(使用jsonify),而且你肯定還會對dict進一步處理,沒必要現在轉化成string,所以通常Flask與Marshmallow配合序列化時,用 dump()
方法即可:
from marshmallow import pprint
user = User(name="Monty", email="monty@python.org") schema = UserSchema() result = schema.dump(user) pprint(result.data) # {"name": "Monty", # "email": "monty@python.org", # "created_at": "2014-08-17T14:54:16.049594+00:00"}
過濾輸出
當然你不需要每次都輸出對象中所有字段,可以使用only
參數來指定你需要輸出的字段,這個在實際場景中很常見:
summary_schema = UserSchema(only=('name', 'email')) summary_schema.dump(user).data # {"name": "Monty Python", "email": "monty@python.org"}
你也可以使用exclude
字段來排除你不想輸出的字段。
Deserializing(反序列化)
相對dump()
的方法就是load()
了,可以將字典等類型轉換成應用層的數據結構,即orm對象:
from pprint import pprint user_data = { 'created_at': '2014-08-11T05:26:03.869245', 'email': u'ken@yahoo.com', 'name': u'Ken' } schema = UserSchema() result = schema.load(user_data) pprint(result.data) # {'name': 'Ken', # 'email': 'ken@yahoo.com', # 'created_at': datetime.datetime(2014, 8, 11, 5, 26, 3, 869245)},
對反序列化而言,將傳入的dict
變成object
更加有意義。在Marshmallow中,dict -> object
的方法需要自己實現,然后在該方法前面加上一個decoration:post_load
即可,即:
from marshmallow import Schema, fields, post_load class UserSchema(Schema): name = fields.Str() email = fields.Email() created_at = fields.DateTime() @post_load def make_user(self, data): return User(**data)
這樣每次調用load()
方法時,會按照make_user
的邏輯,返回一個User
類對象:
user_data = { 'name': 'Ronnie', 'email': 'ronnie@stones.com' } schema = UserSchema() result = schema.load(user_data) result.data # => <User(name='Ronnie')>
tips: 相對於dumps()
,也存在loads()
方法,用於string -> object
,有些簡單場景可以用。
Objects <-> List
上面的序列化和反序列化,是針對一個object而言的,對於objects的處理,只需在schema中增加一個參數:many=True
,即:
user1 = User(name="Mick", email="mick@stones.com") user2 = User(name="Keith", email="keith@stones.com") users = [user1, user2] # option 1: schema = UserSchema(many=True) result = schema.dump(users) # Option 2: schema = UserSchema() result = schema.dump(users, many=True) result.data # [{'name': u'Mick', # 'email': u'mick@stones.com', # 'created_at': '2014-08-17T14:58:57.600623+00:00'} # {'name': u'Keith', # 'email': u'keith@stones.com', # 'created_at': '2014-08-17T14:58:57.600623+00:00'}]
Validation
Schema.load()
和 loads()
方法會在返回值中加入驗證錯誤的dictionary
,例如email
和URL
都有內建的驗證器。
data, errors = UserSchema().load({'email': 'foo'}) errors # => {'email': ['"foo" is not a valid email address.']} # OR, equivalently result = UserSchema().load({'email': 'foo'}) result.errors # => {'email': ['"foo" is not a valid email address.']}
當驗證一個集合時,返回的錯誤dictionary
會以錯誤序號對應錯誤信息的key:value形式保存:
class BandMemberSchema(Schema): name = fields.String(required=True) email = fields.Email() user_data = [ {'email': 'mick@stones.com', 'name': 'Mick'}, {'email': 'invalid', 'name': 'Invalid'}, # invalid email {'email': 'keith@stones.com', 'name': 'Keith'}, {'email': 'charlie@stones.com'}, # missing "name" ] result = BandMemberSchema(many=True).load(user_data) result.errors # {1: {'email': ['"invalid" is not a valid email address.']}, # 3: {'name': ['Missing data for required field.']}}
你可以向內建的field
中傳入validate
參數來定制驗證的邏輯,validate
的值可以是函數,匿名函數lambda
,或者是定義了__call__
的對象:
class ValidatedUserSchema(UserSchema): # NOTE: This is a contrived example. # You could use marshmallow.validate.Range instead of an anonymous function here age = fields.Number(validate=lambda n: 18 <= n <= 40) in_data = {'name': 'Mick', 'email': 'mick@stones.com', 'age': 71} result = ValidatedUserSchema().load(in_data) result.errors # => {'age': ['Validator <lambda>(71.0) is False']}
如果你傳入的函數中定義了ValidationError
,當它觸發時,錯誤信息會得到保存:
from marshmallow import Schema, fields, ValidationError def validate_quantity(n): if n < 0: raise ValidationError('Quantity must be greater than 0.') if n > 30: raise ValidationError('Quantity must not be greater than 30.') class ItemSchema(Schema): quantity = fields.Integer(validate=validate_quantity) in_data = {'quantity': 31} result, errors = ItemSchema().load(in_data) errors # => {'quantity': ['Quantity must not be greater than 30.']}
注意:
如果你需要執行多個驗證,你應該傳入可調用的驗證器的集合(list, tuple, generator)
注意2:
Schema.dump()
也會返回錯誤信息dictionary
,也會包含序列化時的所有ValidationErrors
。但是required
, allow_none
, validate
, @validates
, 和 @validates_schema
只用於反序列化,即Schema.load()
。
Field Validators as Methods
把生成器寫成方法可以提供極大的便利。使用validates
裝飾器就可以注冊一個驗證方法:
from marshmallow import fields, Schema, validates, ValidationError class ItemSchema(Schema): quantity = fields.Integer() @validates('quantity') def validate_quantity(self, value): if value < 0: raise ValidationError('Quantity must be greater than 0.') if value > 30: raise ValidationError('Quantity must not be greater than 30.')
strict Mode
如果將strict=True
傳入Schema
構造器或者class
的Meta
參數里,則僅會在傳入無效數據是報錯。可以使用ValidationError.messages
變量來獲取驗證錯誤的dictionary
。
Required Fields
你可以在field
中傳入required=True
.當Schema.load()
的輸入缺少某個字段時錯誤會記錄下來。
如果需要定制required fields
的錯誤信息,可以傳入一個error_messages
參數,參數的值為以required
為鍵的鍵值對。
class UserSchema(Schema): name = fields.String(required=True) age = fields.Integer( required=True, error_messages={'required': 'Age is required.'} ) city = fields.String( required=True, error_messages={'required': {'message': 'City required', 'code': 400}} ) email = fields.Email() data, errors = UserSchema().load({'email': 'foo@bar.com'}) errors # {'name': ['Missing data for required field.'], # 'age': ['Age is required.'], # 'city': {'message': 'City required', 'code': 400}}
Partial Loading
按照RESTful架構風格的要求,更新數據使用HTTP方法中的PUT
或PATCH
方法,使用PUT方法時,需要把完整的數據全部傳給服務器,使用PATCH
方法時,只需把需要改動的部分數據傳給服務器即可。因此,當使用PATCH
方法時,由於之前設定的required
,傳入數據存在無法通過Marshmallow
數據校驗的風險,為了避免這種情況,需要借助Partial Loading
功能。
實現Partial Loadig
只要在schema
構造器中增加一個partial
參數即可:
class UserSchema(Schema): name = fields.String(required=True) age = fields.Integer(required=True) data, errors = UserSchema().load({'age': 42}, partial=('name',)) # OR UserSchema(partial=('name',)).load({'age': 42}) data, errors # => ({'age': 42}, {})
Schema.validate
如果你只是想用Schema
驗證數據,而不生成對象,可以使用Schema.validate()
.
errors = UserSchema().validate({'name': 'Ronnie', 'email': 'invalid-email'})
errors # {'email': ['"invalid-email" is not a valid email address.']}
Specifying Attribute Names
Schemas
默認會編列傳入對象和自身定義的fields
相同的屬性,然而你也會有需求使用不同的fields
和屬性名。在這種情況下,你需要明確定義這個fields
將從什么屬性名取值:
class UserSchema(Schema): name = fields.String() email_addr = fields.String(attribute="email") date_created = fields.DateTime(attribute="created_at") user = User('Keith', email='keith@stones.com') ser = UserSchema() result, errors = ser.dump(user) pprint(result) # {'name': 'Keith', # 'email_addr': 'keith@stones.com', # 'date_created': '2014-08-17T14:58:57.600623+00:00'}
Specifying Deserialization Keys
Schemas
默認會反編列傳入字典和輸出字典中相同的字段名。如果你覺得數據不匹配你的schema
,你可以傳入load_from
參數指定需要增加load
的字段名(原字段名也能load
,且優先load
原字段名):
class UserSchema(Schema): name = fields.String() email = fields.Email(load_from='emailAddress') data = { 'name': 'Mike', 'emailAddress': 'foo@bar.com' } s = UserSchema() result, errors = s.load(data) #{'name': u'Mike', # 'email': 'foo@bar.com'}
Specifying Serialization Keys
如果你需要編列一個field
成一個不同的名字時,可以使用dump_to
,邏輯和load_from
類似:
class UserSchema(Schema): name = fields.String(dump_to='TheName') email = fields.Email(load_from='CamelCasedEmail', dump_to='CamelCasedEmail') data = { 'name': 'Mike', 'email': 'foo@bar.com' } s = UserSchema() result, errors = s.dump(data) #{'TheName': u'Mike', # 'CamelCasedEmail': 'foo@bar.com'}
“Read-only” and “Write-only” Fields
可以指定某些字段只能夠dump()
或load()
:
class UserSchema(Schema): name = fields.Str() # password is "write-only" password = fields.Str(load_only=True) # created_at is "read-only" created_at = fields.DateTime(dump_only=True)
Nesting Schemas
當你的模型含有外鍵,那這個外鍵的對象在Schemas
如何定義。舉個例子,Blog就具有User對象作為它的外鍵:
Use a Nested field to represent the relationship, passing in a nested schema class. import datetime as dt class User(object): def __init__(self, name, email): self.name = name self.email = email self.created_at = dt.datetime.now() self.friends = [] self.employer = None class Blog(object): def __init__(self, title, author): self.title = title self.author = author # A User object
使用Nested field
表示外鍵對象:
from marshmallow import Schema, fields, pprint class UserSchema(Schema): name = fields.String() email = fields.Email() created_at = fields.DateTime() class BlogSchema(Schema): title = fields.String() author = fields.Nested(UserSchema)
這樣序列化blog就會帶上user信息了:
user = User(name="Monty", email="monty@python.org") blog = Blog(title="Something Completely Different", author=user) result, errors = BlogSchema().dump(blog) pprint(result) # {'title': u'Something Completely Different', # {'author': {'name': u'Monty', # 'email': u'monty@python.org', # 'created_at': '2014-08-17T14:58:57.600623+00:00'}}
如果field 是多個對象的集合,定義時可以使用many
參數:
collaborators = fields.Nested(UserSchema, many=True)
如果外鍵對象是自引用,則Nested里第一個參數為'self'
Specifying Which Fields to Nest
如果你想指定外鍵對象序列化后只保留它的幾個字段,可以使用Only
參數:
class BlogSchema2(Schema): title = fields.String() author = fields.Nested(UserSchema, only=["email"]) schema = BlogSchema2() result, errors = schema.dump(blog) pprint(result) # { # 'title': u'Something Completely Different', # 'author': {'email': u'monty@python.org'} # }
如果需要選擇外鍵對象的字段層次較多,可以使用"."操作符來指定:
class SiteSchema(Schema): blog = fields.Nested(BlogSchema2) schema = SiteSchema(only=['blog.author.email']) result, errors = schema.dump(site) pprint(result) # { # 'blog': { # 'author': {'email': u'monty@python.org'} # } # }
Note
如果你往Nested
是多個對象的列表,傳入only可以獲得這列表的指定字段。
class UserSchema(Schema): name = fields.String() email = fields.Email() friends = fields.Nested('self', only='name', many=True) # ... create ``user`` ... result, errors = UserSchema().dump(user) pprint(result) # { # "name": "Steve", # "email": "steve@example.com", # "friends": ["Mike", "Joe"] # }
這種情況,也可以使用exclude 去掉你不需要的字段。同樣這里也可以使用"."
操作符。
Two-way Nesting
如果有兩個對象需要相互包含,可以指定Nested
對象的類名字符串,而不需要類。這樣你可以包含一個還未定義的對象:
class AuthorSchema(Schema): # Make sure to use the 'only' or 'exclude' params # to avoid infinite recursion books = fields.Nested('BookSchema', many=True, exclude=('author', )) class Meta: fields = ('id', 'name', 'books') class BookSchema(Schema): author = fields.Nested(AuthorSchema, only=('id', 'name')) class Meta: fields = ('id', 'title', 'author')
舉個例子,Author
類包含很多books,而Book
對Author
也有多對一的關系。
from marshmallow import pprint from mymodels import Author, Book author = Author(name='William Faulkner') book = Book(title='As I Lay Dying', author=author) book_result, errors = BookSchema().dump(book) pprint(book_result, indent=2) # { # "id": 124, # "title": "As I Lay Dying", # "author": { # "id": 8, # "name": "William Faulkner" # } # } author_result, errors = AuthorSchema().dump(author) pprint(author_result, indent=2) # { # "id": 8, # "name": "William Faulkner", # "books": [ # { # "id": 124, # "title": "As I Lay Dying" # } # ] # }
Nesting A Schema Within Itself
如果需要自引用,"Nested"構造時傳入"self" (包含引號)即可:
class UserSchema(Schema): name = fields.String() email = fields.Email() friends = fields.Nested('self', many=True) # Use the 'exclude' argument to avoid infinite recursion employer = fields.Nested('self', exclude=('employer', ), default=None) user = User("Steve", 'steve@example.com') user.friends.append(User("Mike", 'mike@example.com')) user.friends.append(User('Joe', 'joe@example.com')) user.employer = User('Dirk', 'dirk@example.com') result = UserSchema().dump(user) pprint(result.data, indent=2) # { # "name": "Steve", # "email": "steve@example.com", # "friends": [ # { # "name": "Mike", # "email": "mike@example.com", # "friends": [], # "employer": null # }, # { # "name": "Joe", # "email": "joe@example.com", # "friends": [], # "employer": null # } # ], # "employer": { # "name": "Dirk", # "email": "dirk@example.com", # "friends": [] # } # }
作者:楊酥餅
鏈接:https://www.jianshu.com/p/594865f0681b
來源:簡書
著作權歸作者所有。商業轉載請聯系作者獲得授權,非商業轉載請注明出處。