Skip to content

Instantly share code, notes, and snippets.

@a-chumagin

a-chumagin/Dockerfile

Last active October 13, 2022 13:46
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save a-chumagin/e5d97a052c07fa5c3fab6fbf38f9eb3d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save a-chumagin/e5d97a052c07fa5c3fab6fbf38f9eb3d to your computer and use it in GitHub Desktop.
Download ZIP
GX quick check result
Raw
Readme.md

run tiny DQ check for example. Do no need datasourses and GX init

  1. pull gist
  2. Build docker image with command docker build -t qdq .
  3. Run check docker run qdq

As result you will see console output same as Console file

Raw
check.py
import pandas as pd
import great_expectations as gx
data = (10, 20, 30, 40, 50, 60, 20, 30, 40, 50, 60, 20, 30, 40, 50, 60, 20, 30, 40, 50, 60.20, 30, 40, 50, 60)
df = pd.DataFrame(data, columns=['Numbers'])
df_ge = gx.from_pandas(df)
df_ge.expect_column_values_to_be_between(column='Numbers', max_value=11, result_format="COMPLETE")
result = df_ge.validate()
print(result)
Raw
console
{
"success": false,
"statistics": {
"evaluated_expectations": 1,
"successful_expectations": 0,
"unsuccessful_expectations": 1,
"success_percent": 0.0
},
"evaluation_parameters": {},
"results": [
{
"success": false,
"exception_info": {
"raised_exception": false,
"exception_message": null,
"exception_traceback": null
},
"expectation_config": {
"meta": {},
"kwargs": {
"column": "Numbers",
"max_value": 11,
"result_format": "COMPLETE"
},
"expectation_type": "expect_column_values_to_be_between"
},
"result": {
"element_count": 25,
"missing_count": 0,
"missing_percent": 0.0,
"unexpected_count": 24,
"unexpected_percent": 96.0,
"unexpected_percent_total": 96.0,
"unexpected_percent_nonmissing": 96.0,
"partial_unexpected_list": [
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.2
],
"partial_unexpected_index_list": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20
],
"partial_unexpected_counts": [
{
"value": 30.0,
"count": 5
},
{
"value": 40.0,
"count": 5
},
{
"value": 50.0,
"count": 5
},
{
"value": 20.0,
"count": 4
},
{
"value": 60.0,
"count": 4
},
{
"value": 60.2,
"count": 1
}
],
"unexpected_list": [
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.0,
20.0,
30.0,
40.0,
50.0,
60.2,
30.0,
40.0,
50.0,
60.0
],
"unexpected_index_list": [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24
]
},
"meta": {}
}
],
"meta": {
"great_expectations_version": "0.15.24",
"expectation_suite_name": "default",
"run_id": {
"run_name": null,
"run_time": "2022-10-12T14:57:04.455326+00:00"
},
"batch_kwargs": {
"ge_batch_id": "226fdcd8-4a3e-11ed-8770-acde48001122"
},
"batch_markers": {},
"batch_parameters": {},
"validation_time": "20221012T145704.455276Z",
"expectation_suite_meta": {
"great_expectations_version": "0.15.24"
}
}
}
Raw
Dockerfile
FROM python:3.7-slim-bullseye
RUN pip install great-expectations==0.15.26
COPY ./*.py /
ENTRYPOINT ["/usr/local/bin/python", "check.py"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment