Relevant test case - Revision history

Robert-schade-e757@uni-paderborn.de at 11:31, 19 July 2024

2024-07-19T11:31:26Z

Alesja-dammer-cf8f@rwth-aachen.de at 11:17, 29 July 2020

2020-07-29T11:17:05Z

Alesja-dammer-cf8f@rwth-aachen.de at 11:16, 29 July 2020

2020-07-29T11:16:38Z

Alesja-dammer-cf8f@rwth-aachen.de at 10:44, 29 July 2020

2020-07-29T10:44:48Z

Paul-kapinos-e26d@rwth-aachen.de at 12:13, 11 May 2020

2020-05-11T12:13:20Z

Paul-kapinos-e26d@rwth-aachen.de at 12:06, 11 May 2020

2020-05-11T12:06:58Z

Paul-kapinos-e26d@rwth-aachen.de at 11:07, 11 May 2020

2020-05-11T11:07:36Z

Paul-kapinos-e26d@rwth-aachen.de at 09:19, 11 May 2020

2020-05-11T09:19:38Z

Paul-kapinos-e26d@rwth-aachen.de at 09:16, 11 May 2020

2020-05-11T09:16:33Z

Paul-kapinos-e26d@rwth-aachen.de at 09:14, 11 May 2020

2020-05-11T09:14:41Z

@@ Line 1: / Line 1: @@
 [[Category:HPC-User]]
 [[Category:HPC-Developer]]
 A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions (to be proven) about real operating point. The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it needs (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course knowledge, which features of software will be used.

@@ Line 8: / Line 8: @@
 * Crop the execution after a handful of iterations (prior reaching the convergence)
 * Omitting some computation parts with known behaviour
 Pay attention to the following:
 * The same software path as in production must be used. If you use in production a compute kernel A, a data set for kernel B would be ''not'' relevant.
 * The hotspots of production runs must also be hotspost in the reduced relevant test case (rule-of-thumb: about half of the overall excution time should be in hotspots). This in turn leads to the rule: Do not downsize too much.

@@ Line 4: / Line 4: @@
 A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions (to be proven) about real operating point. The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it needs (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course knowledge, which features of software will be used.
-In the first approximation  a [set of] real production test cases[s] is a relevant test case for itself, sic! However these test cases are typicall large, long-running and unhandy till impossible to analyse, so a ''reduced'' relevant test case is needed. Typical ways to get a reduced test case are using a real data set and then reduce the size of data set (e.g. grid resolution), crop the execution after a handful of iterations (prior reaching the convergence), or also omitting some computation parts with known behaviour.
+In the first approximation  a [set of] real production test cases[s] is a relevant test case for itself, sic! However these test cases are typicall large, long-running and unhandy to analyse. A ''reduced'' relevant test case is needed. There are some typical ways to get a reduced test case:
-* The same software path as in production must be used. Needless to say that if you use in production a compute kernel A, a data set for kernel B would be ''not'' relevant.
+Pay attention to the following:
-* The hotspots of production runs must also be hotspost in the reduced relevant test case (rule-of-thumb: about of the half of overall excution time should be in hotspots). This in turn lead to the rule: Do not downsize too much.
 * Is the computation numerically stable?
-* Know the scalability pattern of your application depending on data set size: linear, logarithmical, polynomial (O², O³?), or exponentially? (In latter case don't care about scalability - you won't be able to compute interesting data sets until the end of the world..)
+* Know the scalability pattern of your application depending on data set size: linear, logarithmical, polynomial (O², O³?), or exponentially? (In latter case don't care about scalability - you won't be able to compute interesting data sets until the end of the world.)
 * if you accieved any results (e.g. scalability improvement) on a reduced  test case, confirm this with at least selective tests on a full-size test case.

@@ Line 2: / Line 2: @@
 [[Category:HPC-Developer]]
-A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions (to be proven) about real operating point. The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it need (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course which features of software will be used.
+A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions (to be proven) about real operating point. The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it needs (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course knowledge, which features of software will be used.
 In the first approximation  a [set of] real production test cases[s] is a relevant test case for itself, sic! However these test cases are typicall large, long-running and unhandy till impossible to analyse, so a ''reduced'' relevant test case is needed. Typical ways to get a reduced test case are using a real data set and then reduce the size of data set (e.g. grid resolution), crop the execution after a handful of iterations (prior reaching the convergence), or also omitting some computation parts with known behaviour.

@@ Line 1: / Line 1: @@
 [[Category:HPC-User]]
 [[Category:HPC-Developer]]
 A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions (to be proven) about real operating point. The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it need (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course which features of software will be used.
-In the first approximation  a [set of] real production test cases[s] is a relevant test case for itself, sic! However these test cases are typicall large, long-running and unhandy till impossible to analyse, so a ''reduced'' relevant test case is needed. Typical ways to get a reduced test case are using a real data set and then reduce the size of data set (e.g. grid resolution), crop the execution after a handful of iterations (prior reaching the convergence), or also elimination some computation parts with known behaviour.
+In the first approximation  a [set of] real production test cases[s] is a relevant test case for itself, sic! However these test cases are typicall large, long-running and unhandy till impossible to analyse, so a ''reduced'' relevant test case is needed. Typical ways to get a reduced test case are using a real data set and then reduce the size of data set (e.g. grid resolution), crop the execution after a handful of iterations (prior reaching the convergence), or also omitting some computation parts with known behaviour.
 * The same software path as in production must be used. Needless to say that if you use in production a compute kernel A, a data set for kernel B would be ''not'' relevant.
@@ Line 12: / Line 10: @@
 * Is the computation numerically stable?
 * Know the scalability pattern of your application depending on data set size: linear, logarithmical, polynomial (O², O³?), or exponentially? (In latter case don't care about scalability - you won't be able to compute interesting data sets until the end of the world..)

← Older revision		Revision as of 09:19, 11 May 2020
Line 4:		Line 4:


−	A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions about ~~true production state~~ (to be proven). The definition of a relevant test case is essential and vital for performance engeneering. Typically it need (at least basic) knowledge about algorithms used in the application, ~~preduction~~ about needed size of computation jobs, and of course which features of software will be used.	+	A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions about real operating point (to be proven). The definition of a relevant test case is essential and vital for performance engeneering both for developing an application and for efficient use of (even blackboxed) application. Typically it need (at least basic) knowledge about algorithms used in the application, prediction about needed size of computation jobs, and of course which features of software will be used.

	In the first approximation a [set of] real production test cases[s] is a relevant test case for himself, sic! However these test cases are typicall large, long-running and unhandy to impossible to analyse, so a ''reduced'' relevant test case is needed.		In the first approximation a [set of] real production test cases[s] is a relevant test case for himself, sic! However these test cases are typicall large, long-running and unhandy to impossible to analyse, so a ''reduced'' relevant test case is needed.

← Older revision		Revision as of 09:16, 11 May 2020
Line 1:		Line 1:
		+	[[Category:HPC-User]]
		+	[[Category:HPC-Developer]]
		+
		+
		+
	A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions about true production state (to be proven). The definition of a relevant test case is essential and vital for performance engeneering. Typically it need (at least basic) knowledge about algorithms used in the application, preduction about needed size of computation jobs, and of course which features of software will be used.		A relevant test case is a combination of data set, application and parameters which reflects production behaviour or at least allow assumptions about true production state (to be proven). The definition of a relevant test case is essential and vital for performance engeneering. Typically it need (at least basic) knowledge about algorithms used in the application, preduction about needed size of computation jobs, and of course which features of software will be used.

← Older revision		Revision as of 09:14, 11 May 2020
Line 4:		Line 4:

	* The same software path as in production must be used. Needless to say that if you use in production a compute kernel A, a data set for kernel B would be ''not'' relevant.		* The same software path as in production must be used. Needless to say that if you use in production a compute kernel A, a data set for kernel B would be ''not'' relevant.
−	* The hotspots of production runs must also be hotspost in	+	* The hotspots of production runs must also be hotspost in the reduced relevant test case (rule-of-thumb: about of the half of overall excution time should be in hotspots). This in turn lead to the rule: Do not downsize too much (for performance engineering